In this paper, we consider a class of Forward-Backward (FB) splitting methods that includes several variants (e.g. inertial schemes, FISTA) for minimizing the sum of two proper convex and lower semi-continuous functions, one of which has a Lipschitz continuous gradient, and the other is partly smooth relative to a smooth active manifold M. We propose a unified framework, under which we show that, this class of FB-type algorithms (i) correctly identifies the active manifolds in a finite number of iterations (finite activity identification), and (ii) then enters a local linear convergence regime, which we characterize precisely in terms of the structure of the underlying active manifolds. For simpler problems involving polyhedral functions, we show finite termination. We also establish and explain why FISTA (with convergent sequences) locally oscillates and can be slower than FB. These results may have numerous applications including in signal/image processing, sparse recovery and machine learning. Indeed, the obtained results explain the typical behaviour that has been observed numerically for many problems in these fields such as the Lasso, the group Lasso, the fused Lasso and the nuclear norm minimization to name only a few. Forward-Backward-type splitting methodsThe Forward-Backward (FB) splitting method [40] is a powerful tool for solving optimization problems (P opt ) with the additively separable and "smooth + non-smooth" structure. The standard (non-relaxed) version of FB updates a new iterate x k+1 based on the following rule, (x 0 ∈ R n is chosen arbitrarily)( 1.1) where , > 0, and prox γR denotes the proximity operator of R which is defined asThe scheme (1.1) recovers the gradient descent method when R = 0, and the classic Proximal Point Algorithm (PPA) [53] when F = 0. Global convergence of the sequence (x k ) k∈N generated by FB method is well established in the literature, based on the property that the composed operator prox γR (Id − γ∇F ) is so-called averaged non-expansive [12]. Moreover, sub-linear O(1/k) convergence rate of the sequence of objective values of FB is also established in e.g. [47,16,14].Inertial schemes and FISTA In the literature, different variants of the FB method were studied, and a popular trend is the inertial schemes which aim to speed up the convergence property of FB. In [51], a twostep algorithm called the "heavy-ball with friction" method is studied for solving (P opt ) with R = 0. It can be seen as an explicit discretization of a nonlinear second-order dynamical system (oscillator with viscous damping). This dynamical approach to iterative methods in optimization has motivated increasing attention in recent years. For instance, in real Hilbert spaces, it is used in [4] for solving (P opt ) with F = 0 and [5] for solving (P inc ) with B = 0 yielding an intertial PPA method. The authors in [44,8,41] propose different inertial versions of the FB method for solving (P opt ) and/or (P inc ) in real Hilbert spaces.On the other hand, in the context of convex optimization, the a...

In this paper, we present a convergence rate analysis for the inexact Krasnosel'skiȋ-Mann iteration built from non-expansive operators. The presented results include two main parts: we first establish the global pointwise and ergodic iteration-complexity bounds; then, under a metric sub-regularity assumption, we establish a local linear convergence for the distance of the iterates to the set of fixed points. The obtained results can be applied to analyze the convergence rate of various monotone operator splitting methods in the literature, including the Forward-Backward splitting, the Generalized Forward-Backward, the Douglas-Rachford splitting, alternating direction method of multipliers (ADMM) and Primal-Dual splitting methods. For these methods, we also develop easily verifiable termination criteria for finding an approximate solution, which can be seen as a generalization of the termination criterion for the classical gradient descent method. We finally develop a parallel analysis for the non-stationary Krasnosel'skiȋ-Mann iteration.1 In fact, in many cases the fixed-point operator T is even α-averaged, see Definition 1. 2 The authors consider the case where H is any normed space. 2 where J γA1 = (Id + γA 1 ) −1 is the resolvent of A 1 . Relying on firm non-expansiveness of the resolvent [6], [33] have shown that ||e k || = O(1/ √ k). In fact, it is easy to show that (3) is equivalent tois firmly non-expansive, and thus fits in our framework. Our results in Section 3 go much beyond this by considering a more general iterative scheme with an operator that is only non-expansive and may be evaluated approximately.Relation with HPE Based on the enlargement of maximal monotone operators, in [62], a hybrid proximal extragradient method (HPE) is introduced to solve monotone inclusion problems of the formThe HPE framework encompasses some splitting algorithms in the literature [48]. The convergence of HPE is established in [62] and in [13] for its inexact version. The pointwise and ergodic iteration-complexities of the exact HPE on a similar error criterion as in our work are established in [48]. Some of the splitting methods we consider in Section 5 are also covered by HPE, hence our iteration-complexity bounds coincide with those of HPE, but only for the ergodic case. While for the pointwise case, our bound is uniform and theirs is not (see further discussion in Remark 3). Local linear convergenceRelation with [38,40] In [38], local linear convergence of the distance to the set of zeros of a maximal monotone operator using the exact proximal point algorithm (PPA [45,60]) is established by assuming metric sub-regularity of the operator. Local convergence rate analysis of PPA under a higher-order extension of metric sub-regularity, namely metric q-sub-regularity q ∈]0, 1], is conducted in [40]. In our work, metric sub-regularity is assumed on Id − T with T being the fixed-point operator, i.e. the resolvent, rather than the maximal monotone operator in the case of PPA. Relation between metric sub-regularity of these ...

The Douglas-Rachford and alternating direction method of multipliers are two proximal splitting algorithms designed to minimize the sum of two proper lower semi-continuous convex functions whose proximity operators are easy to compute. The goal of this work is to understand the local linear convergence behaviour of Douglas-Rachford (resp. alternating direction method of multipliers) when the involved functions (resp. their Legendre-Fenchel conjugates) are moreover partly smooth. More precisely, when the two functions (resp. their conjugates) are partly smooth relative to their respective smooth submanifolds, we show that Douglas-Rachford (resp. alternating direction method of multipliers) (i) identifies these manifolds in finite time; (ii) enters a local linear convergence regime. When both functions are locally polyhedral, we show that the optimal convergence radius is given in terms of the cosine of the Friedrichs angle between the tangent spaces of the identified submanifolds. Under polyhedrality of both functions, we also provide conditions sufficient for finite convergence. The obtained results are illustrated by several concrete examples and supported by numerical experiments.