A First-Order Stochastic Primal-Dual Algorithm with Correction Step

Abstract

We investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in [26,35] for saddle point problems. It consists in a forward step determined by the stochastic evaluation of the cocoercive operator, a backward step in the dual variables involving the resolvent of the monotone operator, and an additional forward step using the stochastic evaluation of the cocoercive introduced in the first step. We prove weak almost sure convergence of the iterates by showing that the primal-dual sequence generated by the method is stochastic quasi Fejér-monotone with respect to the set of zeros of the considered primal and dual inclusions. Additional results on ergodic convergence in expectation are considered for the special case of saddle point models.

A Stochastic Variance Reduction Algorithm with Bregman Distances for Structured Composite Problems

“…Remark 3.7 Recently, there appeared various publications where the convex-concave saddle problems were investigated in stochastic setting; see [3,12,14,21,33,34,24,30,27,36,41,42] for instances and the reference therein. These existing methods are different from our proposed algorithm.…”

“…In other words, they are full-splitting. The stochastic counterparts of some primal-dual splitting methods were also investigated in the literature; see [33,21,14,34] for instances.…”

Convergence analysis of the stochastic reflected forward-backward splitting algorithm

“…In the general case, to ensure the weak almost sure convergence, we not only need the step-size bounded away from 0 but also the summable condition in (3.17). These conditions were used in [7,9,29,30].…”

Variable Smoothing for Convex Optimization Problems Using Stochastic Gradients

“…For the finite sum case (2), there exist algorithms of similar spirit such as those in [14,24]. Some algorithms do in fact deal with a similar setup of stochastic gradient like evaluations, see [26], but only for smooth terms in the objective function.…”