Line search for generalized alternating projections

Abstract

This paper is about line search for the generalized alternating projections (GAP) method. This method is a generalization of the von Neumann alternating projections method, where instead of performing alternating projections, relaxed projections are alternated. The method can be interpreted as an averaged iteration of a nonexpansive mapping. Therefore, a recently proposed line search method for such algorithms is applicable to GAP. We evaluate this line search and show situations when the line search can be performed with little additional cost. We also present a variation of the basic line search for GAP-the projected line search. We prove its convergence and show that the line search condition is convex in the step length parameter. We show that almost all convex optimization problems can be solved using this approach and numerical results show superior performance with both the standard and the projected line search, sometimes by several orders of magnitude, compared to the nominal method.

Optimal convergence rates for generalized alternating projections

“…Thus α k+1 ∈ (0, 2) and each iteration is the result of an averaged mapping S k with fixed points U ∩ V. It follows that the iterates converge to the fixed point set U ∩ V, see e.g. [25].…”

“…Let the relaxed projection onto a set C, with relaxation parameter α, be defined as P α C := (1 − α)I + αP C . The generalized alternating projections (GAP) [25] for two closed, convex and nonempty sets U and V, with U ∩ V = ∅, is then defined by the iteration…”

“…The operator S is averaged and the iterates converge to the fixed-point set fixS under the following assumption, see e.g. [25] where these results are collected.…”

Chordal decomposition in operator-splitting methods for sparse semidefinite programs

“…We remark that the current implementation of our algorithms is sequential, but many steps can be carried out in parallel, so further computational gains may be achieved by taking full advantage of distributed computing architectures. Besides, it would be interesting to integrate some acceleration techniques (e.g., [15,41]) that promise to improve the convergence performance of ADMM in practice.…”

“…, λ p }. According to (4), each iteration of the ADMM requires the minimization of the Lagrangian in (15) with respect to the X -and Y-blocks separately, followed by an update of the multipliers Z. At each step, the variables not being optimized over are fixed to their most current value.…”