Convergence Rates of Forward–Douglas–Rachford Splitting Method

Molinari, Cesare; Liang, Jingwei; Fadili, Jalal M.

doi:10.1007/s10957-019-01524-9

Cited by 10 publications

(4 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the past, composite, constrained problems like (P) have been approached using proximal splitting methods, e.g. generalized forward-backward as developed in [32] or forward-douglas-rachford [25]. Such approaches require one to compute the proximal mapping associated to the function h. Alternatively, when the objective function satisfies some regularity conditions and when the constraint set is well behaved, one can forgo computing a proximal mapping, instead computing a linear minimization oracle.…”

Section: Relation To Prior Workmentioning

confidence: 99%

Generalized Conditional Gradient with Augmented Lagrangian for Composite Minimization

Silveti-Falls¹,

Molinari²,

Fadili³

2020

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

In this paper we propose a splitting scheme which hybridizes generalized conditional gradient with a proximal step which we call CGALP algorithm, for minimizing the sum of three proper convex and lower-semicontinuous functions in real Hilbert spaces. The minimization is subject to an affine constraint, that allows in particular to deal with composite problems (sum of more than three functions) in a separate way by the usual product space technique. While classical conditional gradient methods require Lipschitz-continuity of the gradient of the differentiable part of the objective, CGALP needs only differentiability (on an appropriate subset), hence circumventing the intricate question of Lipschitz continuity of gradients. For the two remaining functions in the objective, we do not require any additional regularity assumption. The second function, possibly nonsmooth, is assumed simple, i.e., the associated proximal mapping is easily computable. For the third function, again nonsmooth, we just assume that its domain is weakly compact and that a linearly perturbed minimization oracle is accessible. In particular, this last function can be chosen to be the indicator of a nonempty bounded closed convex set, in order to deal with additional constraints. Finally, the affine constraint is addressed by the augmented Lagrangian approach. Our analysis is carried out for a wide choice of algorithm parameters satisfying so called "open loop" rules. As main results, under mild conditions, we show asymptotic feasibility with respect to the affine constraint, boundedness of the dual multipliers, and convergence of the Lagrangian values to the saddle-point optimal value. We also provide (subsequential) rates of convergence for both the feasibility gap and the Lagrangian values.

show abstract

Section: Relation To Prior Workmentioning

confidence: 99%

Generalized Conditional Gradient with Augmented Lagrangian for Composite Minimization

Silveti-Falls¹,

Molinari²,

Fadili³

2020

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

show abstract

“…L . Nevertheless, according to the reference [24], for the objective Equation (3), the convergence rate of f (…”

Section: Fast Iterative Soft Threshold Function Reconstruction Algori...mentioning

confidence: 99%

A Faster and More Accurate Iterative Threshold Algorithm for Signal Reconstruction in Compressed Sensing

Wei

Mao

Dai

et al. 2022

Sensors

View full text Add to dashboard Cite

Fast iterative soft threshold algorithm (FISTA) is one of the algorithms for the reconstruction part of compressed sensing (CS). However, FISTA cannot meet the increasing demands for accuracy and efficiency in the signal reconstruction. Thus, an improved algorithm (FIPITA, fast iterative parametric improved threshold algorithm) based on mended threshold function, restart adjustment mechanism and parameter adjustment is proposed. The three parameters used to generate the gradient in the FISTA are carefully selected by assessing the impact of them on the performance of the algorithm. The developed threshold function is used to replace the soft threshold function to reduce the reconstruction error and a restart mechanism is added at the end of each iteration to speed up the algorithm. The simulation experiment is carried out on one-dimensional signal and the FISTA, RadaFISTA and RestartFISTA are used as the comparison objects, with the result that in one case, for example, the residual rate of FIPITA is about 6.35% lower than those three and the number of iterations required to achieve the minimum error is also about 102 less than that of FISTA.

show abstract

“…Identifiability is also exploited as tool for algorithmic design for the so called "UV-algorithms" [46]. Recent works [47,36,37] study finite time identification for popular FOMs. In particular, Liang et al [37] show that the iterates of PDHG identify the active constraints in finite time, provided the limit point is nondegenerate.…”

Section: Related Workmentioning

confidence: 99%

“…Before we describe these statistics, let us define what we mean by approximate infeasibility certificates. The set of (exact) primal infeasibility certificates for (47) is given by all the vectors (y, r) ∈ R m + × R n satisfying b y + l r + − u r − > 0 and r = −A y (48) while the set of (exact) dual infeasibility certificates is given by all the vectors x ∈ R n satisfying c x < 0, x ∈ C v , and Ax ≥ 0 where the set C v is given by…”

Section: Numerical Experimentsmentioning

confidence: 99%

Infeasibility detection with primal-dual hybrid gradient for large-scale linear programming

Applegate,

Díaz,

et al. 2021

Preprint

View full text Add to dashboard Cite

We study the problem of detecting infeasibility of large-scale linear programming problems using the primal-dual hybrid gradient method (PDHG) of . The literature on PDHG has mostly focused on settings where the problem at hand is assumed to be feasible. When the problem is not feasible, the iterates of the algorithm do not converge. In this scenario, we show that the iterates diverge at a controlled rate towards a well-defined ray. The direction of this ray is known as the infimal displacement vector v. The first contribution of our work is to prove that this vector recovers certificates of primal and dual infeasibility whenever they exist. Based on this fact, we propose a simple way to extract approximate infeasibility certificates from the iterates of PDHG. We study three different sequences that converge to the infimal displacement vector: the difference of iterates, the normalized iterates, and the normalized average. All of them are easy to compute, and thus the approach is suitable for large-scale problems. Our second contribution is to establish tight convergence rates for these sequences. We demonstrate that the normalized iterates and the normalized average achieve a convergence rate of O 1 k , improving over the known rate of O 1 √ k . This rate is general and applies to any fixed-point iteration of a nonexpansive operator. Thus, it is a result of independent interest since it covers a broad family of algorithms, including, for example, ADMM, and can be applied settings beyond linear programming, such as quadratic and semidefinite programming. Further, in the case of linear programming we show that, under nondegeneracy assumptions, the iterates of PDHG identify the active set of an auxiliary feasible problem in finite time, which ensures that the difference of iterates exhibits eventual linear convergence to the infimal displacement vector.

show abstract

Convergence Rates of Forward–Douglas–Rachford Splitting Method

Cited by 10 publications

References 45 publications

Generalized Conditional Gradient with Augmented Lagrangian for Composite Minimization

Generalized Conditional Gradient with Augmented Lagrangian for Composite Minimization

A Faster and More Accurate Iterative Threshold Algorithm for Signal Reconstruction in Compressed Sensing

Infeasibility detection with primal-dual hybrid gradient for large-scale linear programming

Contact Info

Product

Resources

About