On the Optimal Linear Convergence Rate of a Generalized Proximal Point Algorithm

Abstract

Abstract. The proximal point algorithm (PPA) has been well studied in the literature. In particular, its linear convergence rate has been studied by Rockafellar in 1976 under certain condition. We consider a generalized PPA in the generic setting of finding a zero point of a maximal monotone operator, and show that the condition proposed by Rockafellar can also sufficiently ensure the linear convergence rate for this generalized PPA. Indeed we show that these linear convergence rates are optimal.Both the exact and inexact versions of this generalized PPA are discussed. The motivation to consider this generalized PPA is that it includes as special cases the relaxed versions of some splitting methods that are originated from PPA. Thus, linear convergence results of this generalized PPA can be used to better understand the convergence of some widely used algorithms in the literature. We focus on the particular convex minimization context and specify Rockafellar's condition to see how to ensure the linear convergence rate for some efficient numerical schemes, including the classical augmented Lagrangian method proposed by Hensen and Powell in 1969 and its relaxed version, the original alternating direction method of multipliers (ADMM) by Glowinski and Marrocco in 1975 and its relaxed version (i.e., the generalized ADMM by Eckstein and Bertsekas in 1992).Some refined conditions weaker than existing ones are proposed in these particular contexts.

Weak and linear convergence of a generalized proximal point algorithm with alternating inertial steps for a monotone inclusion problem

“…The above definition was given in [8]. The example of a set-valued map T such that T −1 is Lipschitz continuous at 0 ws given in [11]. It also was shown in [11] that the Lipschitz continuity at 0 is weaker than the strong monotonicity assumed in [23,34].…”

“…Lemma 2.1. (see [11]) Let T : H → 2 H be set-valued and maximal monotone, and define J cT := (I + cT ) −1 with c > 0. Then, (i) J cT (z) − J cT (z ), (I…”

“…It turns out that the PPA is a very powerful algorithmic tool and includes many known algorithms, such as, the classical augmented Lagrangian method, the Douglas-Rachford splitting method, and the alternating direction method of multipliers as special cases. For more facts about the PPA and its generalizations, we refer to [9,10,11].…”

“…It has been shown that with the alternated iPPA, some sort of Fejér monotonicity of z k − z * is recovered and that the method out-performs their non-inertial counterparts; see, e.g., [28,31,32] for more details. In [11], Tao and Yuan studied the generalized PPA for a maximal monotone set-valued operator T with T −1 being Lipschitz continuous at 0 in real Hilbert space, and obtained both weak and linear convergence results.…”

Iteration-complexity analysis of a generalized alternating direction method of multipliers

“…Some strategies are also proposed in order to choose the relaxation and penalty parameters. Linear convergence of the G-ADMM is also studied in [31] on a general setting. Paper [30] studies the G-ADMM as a particular case of a general scheme in a Hilbert space and measures, in an ergodic sense, a "partial" primal-dual gap associated to the augmented Lagrangian of problem (1).…”