Activity Identification and Local Linear Convergence of Forward--Backward-type Methods

Liang, Jingwei; Fadili, Jalal M.; Peyré, Gabriel

doi:10.1137/16m106340x

Cited by 82 publications

(143 citation statements)

References 47 publications

Supporting

Mentioning

138

Contrasting

Order By: Relevance

“…We further give bounds, under the assumptions above and the standard nondegeneracy condition, on the active-set complexity of the proximal gradient method. We are only aware of one previous work giving such bounds, the work of Liang et al who included a bound on the active-set complexity of the proximal gradient method [17,Proposition 3.6]. Unlike this work, their result does not evoke strong-convexity.…”

Section: Motivationmentioning

confidence: 92%

“Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern?

2018

View full text Add to dashboard Cite

Proximal gradient methods have been found to be highly effective for solving minimization problems with non-negative constraints or ℓ 1regularization. Under suitable nondegeneracy conditions, it is known that these algorithms identify the optimal sparsity pattern for these types of problems in a finite number of iterations. However, it is not known how many iterations this may take. We introduce the notion of the "active-set complexity", which in these cases is the number of iterations before an algorithm is guaranteed to have identified the final sparsity pattern. We further give a bound on the active-set complexity of proximal gradient methods in the common case of minimizing the sum of a strongly-convex smooth function and a separable convex non-smooth function.

show abstract

Section: Motivationmentioning

confidence: 92%

“Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern?

2018

View full text Add to dashboard Cite

show abstract

“…However, there exist accelerated versions such as FISTA [4], which benefit from a better non-asymptotic rate of convergence (O(1/k 2 )). Finally, it is noteworthy that these proximal methods enjoy a linear asymptotic rate (see for instance [64]), but this regime can be slow to reach.…”

Section: Solving the Blassomentioning

confidence: 99%

The sliding Frank–Wolfe algorithm and its application to super-resolution microscopy

Denoyelle¹,

Duval²,

Peyré³

et al. 2019

Inverse Problems

Self Cite

167

View full text Add to dashboard Cite

This paper showcases the theoretical and numerical performance of the Sliding Frank-Wolfe, which is a novel optimization algorithm to solve the BLASSO sparse spikes super-resolution problem.The BLASSO is a continuous (i.e. off-the-grid or grid-less) counterpart to the well-known 1 sparse regularisation method (also known as LASSO or Basis Pursuit). Our algorithm is a variation on the classical Frank-Wolfe (also known as conditional gradient) which follows a recent trend of interleaving convex optimization updates (corresponding to adding new spikes) with non-convex optimization steps (corresponding to moving the spikes). Our main theoretical result is that this algorithm terminates in a finite number of steps under a mild non-degeneracy hypothesis. We then target applications of this method to several instances of single molecule fluorescence imaging modalities, among which certain approaches rely heavily on the inversion of a Laplace transform. Our second theoretical contribution is the proof of the exact support recovery property of the BLASSO to invert the 1-D Laplace transform in the case of positive spikes. On the numerical side, we conclude this paper with an extensive study of the practical performance of the Sliding Frank-Wolfe on different instantiations of single molecule fluorescence imaging, including convolutive and non-convolutive (Laplace-like) operators. This shows the versatility and superiority of this method with respect to alternative sparse recovery technics.

show abstract

“…(iii) Theorem 3.3 only states the existence of K after which the identification of the sequences happens, and no bounds are available. In [36,37], lower bounds of K for the FB and DR splitting methods are provided, and similar lower bounds can be obtained here for the Primal-Dual splitting methods. Since such lower-bounds are only of theoretical interest, we decided to skip the corresponding details here and refer the reader to [36,37].…”

mentioning

confidence: 70%

“…The reason behind this is that in the exact case, under condition (ND), the proximal mapping of the partly smooth function R and that of its restriction to M R x locally agree nearby x (and similarly for J * and v ). This property can be easily violated if approximate proximal mappings are involved, see [36] for an example. (iii) Theorem 3.3 only states the existence of K after which the identification of the sequences happens, and no bounds are available.…”

mentioning

confidence: 99%

Local linear convergence analysis of Primal–Dual splitting methods

2018

Self Cite

View full text Add to dashboard Cite

In this paper, we study the local linear convergence properties of a versatile class of Primal-Dual splitting methods for minimizing composite non-smooth convex optimization problems. Under the assumption that the non-smooth components of the problem are partly smooth relative to smooth manifolds, we present a unified local convergence analysis framework for these Primal-Dual splitting methods. More precisely, in our framework we first show that (i) the sequences generated by Primal-Dual splitting methods identify a pair of primal and dual smooth manifolds in a finite number of iteration, and then (ii) enter a local linear convergence regime, which is for instance characterized in terms of the structure of the underlying active smooth manifolds. We also show how our results for Primal-Dual splitting specialize to cover existing one on Forward-Backward splitting and Douglas-Rachford splitting/ADMM (alternating direction methods of multipliers). Moreover, based on these obtained local convergence analysis result, several practical acceleration techniques for the class of Primal-Dual splitting methods are discussed. To exemplify the usefulness of the obtained result, we consider several concrete numerical experiments arising from applicative fields including signal/image processing, inverse problems and machine learning, etc. The demonstration not only verify the local linear convergence behaviour of Primal-Dual splitting methods, but also the insights on how to accelerate them in practice. Primal-Dual splitting methodsPrimal-Dual splitting methods to solve more or less complex variants of (P P )-(P D ) have witnessed a recent wave of interest in the literature [13,11,50,18,29,21,16]. All these approaches achieve full splitting, they involve the resolvents of R and J * , the gradients of F and G * and the linear operator L, all separately at various points in the course of iteration. For instance, building on the seminal work of [3], the now-popular scheme proposed in [13] solves (P P )-(P D ) with F = G * = 0. The authors in [29] have shown that the Primal-Dual splitting method of [13] can be seen as a proximal point algorithm (PPA) in R n × R m endowed with a suitable norm. Exploiting the same idea, the author in [21] considered (P P ) with G * = 0, and proposed an iterative scheme which can be interpreted as a Forward-Backward (FB) splitting again with an appropriately renormed space. This idea is further extended in [50] to solve more complex problems such as that in (P P ). A variable metric version was proposed in [19]. Motivated by the structure of (1.1), [11] and [18] proposed a Forward-Backward-Forward scheme [48] to solve it.In this paper, we will focus the unrelaxed Primal-Dual splitting method summarized in Algorithm 1. This scheme covers that of [13,50,29,21,16]. Though we omit the details here for brevity, our analysis and conclusions carry through to the method proposed in [11,18]. Algorithm 1: A Primal-Dual splitting method(1.2) k = k + 1; until convergence;See Section A.2 for the proof. Remark 3.4....

show abstract

Activity Identification and Local Linear Convergence of Forward--Backward-type Methods

Cited by 82 publications

References 47 publications

“Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern?

“Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern?

The sliding Frank–Wolfe algorithm and its application to super-resolution microscopy

Local linear convergence analysis of Primal–Dual splitting methods

Contact Info

Product

Resources

About