Asynchronous block-iterative primal-dual decomposition methods for monotone inclusions

Abstract

We propose new primal-dual decomposition algorithms for solving systems of inclusions involving sums of linearly composed maximally monotone operators. The principal innovation in these algorithms is that they are block-iterative in the sense that, at each iteration, only a subset of the monotone operators needs to be processed, as opposed to all operators as in established methods. Deterministic strategies are used to select the blocks of operators activated at each iteration. In addition, we allow for operator processing "lags", permitting asynchronous implementation. The decomposition phase of each iteration of our methods is to generate points in the graphs of the selected monotone operators, in order to construct a half-space containing the Kuhn-Tucker set associated with the system. The coordination phase of each iteration involves a projection onto this half-space. We present two related methods: the first method provides weakly convergent primal and dual sequences under general conditions, while the second is a variant in which strong convergence is guaranteed without additional assumptions. Neither algorithm requires prior knowledge of bounds on the linear operators involved or the inversion of linear operators. Our algorithmic framework unifies and significantly extends the approaches taken in earlier work on primal-dual projective splitting methods.

Projective splitting with forward steps only requires continuity

“…Then the sequence {(z k , w k )} generated by Algorithm 1 converges weakly to some point (z, w) in the extended solution set S of (3) defined in (5). Furthermore, x k i ⇀ G iz and y k i ⇀ w i for all i = 1, .…”

“…Clearly z ∈ H 0 solves (3) if and only if there exists w ∈ H 1 ×· · ·×H n−1 such that (z, w) ∈ S. Algorithm 1 is a special case of a general seperator-projector method for finding a point in a closed and convex set. At each iteration the method constructs an affine function ϕ k : H n → R which separates the current point from the target set S defined in (5). In other words, if p k is the current point in H generated by the algorithm, ϕ k (p k ) > 0, and ϕ k (p) ≤ 0 for all p ∈ S. The next point is then the projection of p k onto the hyperplane {p : ϕ k (p) = 0}, subject to a relaxation factor β k .…”

Convergence Rates for Projective Splitting

“…For (2), we establish an ergodic O(1/k) function value convergence rate for iterates generated by projective splitting. (1) is strongly monotone, we establish strong convergence, rather than weak, in the general Hilbert space setting, without using the Haugazeau [13] modification employed to obtain general strong convergence in [4]. Furthermore, we derive an ergodic O(1/ √ k) convergence rate for the distance of the iterates to the unique solution of (1).…”

Efficient Constrained Signal Reconstruction by Randomized Epigraphical Projection

“…Finally, applying Cardano formula to (15) yields (13). 3 Algorithm 1: Proposed algorithm for solving (5) input :…”

“…Recently, stochastic primal-dual splitting algorithms have been intensively studied for stochastic optimization [11][12][13][14]. Roughly speaking, at each iteration, the algorithms activate only the operations associated with randomly chosen variables, so that the said costs are significantly reduced.…”