Recent Advances in Variable Metric First-Order Methods

Bonettini, Silvia; Porta, Federica; Prato, Marco; Rebegoldi, Simone; Ruggiero, Valeria; Zanni, Luca

doi:10.1007/978-3-030-32882-5_1

Cited by 15 publications

(21 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Strategies of choosing the step sizes α k and the matrices D k have their origin in the study of the gradient method for unconstrained optimization, papers dealing with this issue include but are not limited to [7,27,29,36,69], see also [18,25,26,49]. More details about selecting step sizes α k and matrices D k can be found in the recent review [17] and references therein.…”

Section: Inexact Scaled Gradient Methodsmentioning

confidence: 99%

“…Proof. Using the first inequality in (17) and Lemma 19, we have 2α min τ min ≤ 2α k τ k , for all k ∈ N.…”

Section: Iteration-complexity Boundmentioning

confidence: 99%

“…After these works, several variants of it have appeared over the years, resulting in a vast literature on the subject, including [10,11,12,33,35,40,47,56,67]. Additional reference on this subject can be found in the recent review [17] and references therein. Among all the variants of the gradient projection method, the scaled version has been especially considered due to the flexibility provided in efficient implementations of the method; see [13,5,16,18,19].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the inexact scaled gradient projection method

Ferreira¹,

Lemes²,

Prudente³

2021

Preprint

View full text Add to dashboard Cite

The purpose of this paper is to present an inexact version of the scaled gradient projection method on a convex set, which is inexact in two sense. First, an inexact projection on the feasible set is computed, allowing for an appropriate relative error tolerance. Second, an inexact non-monotone line search scheme is employed to compute a step size which defines the next iteration. It is shown that the proposed method has similar asymptotic convergence properties and iteration-complexity bounds as the usual scaled gradient projection method employing monotone line searches.

show abstract

Section: Inexact Scaled Gradient Methodsmentioning

confidence: 99%

“…Proof. Using the first inequality in (17) and Lemma 19, we have 2α min τ min ≤ 2α k τ k , for all k ∈ N.…”

Section: Iteration-complexity Boundmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the inexact scaled gradient projection method

Ferreira¹,

Lemes²,

Prudente³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…• the use of a variable metric in (3), which is induced along the iterations by a sequence of linear, bounded and self-adjoint positive operators {D k } k∈N , typically chosen so as to capture second order information of the differentiable part f at the current iterate x (k) (see e.g. [4], [6], [7], [14]); • the inexact computation of the proximal-gradient point according to Definition 1;…”

Section: A Inexact S-fistamentioning

confidence: 99%

A scaled, inexact and adaptive Fast Iterative Soft-Thresholding Algorithm for convex image restoration

Calatroni¹,

Rebegoldi²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this note, we consider a special instance of the scaled, inexact and adaptive generalised Fast Iterative Soft-Thresholding Algorithm (SAGE-FISTA) recently proposed in [15] for the efficient solution of strongly convex composite optimisation problems. In particular, we address here the sole (non-strongly) convex optimisation scenario, which is frequently encountered in many imaging applications. The proposed inexact S-FISTA algorithm shows analogies to the variable metric and inexact version of FISTA studied in [6], the main difference being the use of an adaptive (non-monotone) backtracking strategy allowing for the automatic adjustment of the algorithmic step-size along the iterations (see [8], [17]). A quadratic convergence result in function values depending on the backtracking parameters and the upper and lower bounds on the spectrum of the variable metric operators is given. Experimental results on TV image deblurring problems with Poisson noise are then reported for numerical validation, showing improved computational efficiency and precision.

show abstract

“…In the following, we describe a generalized version of the popular "Fast Iterative Soft-Thresholding Algorithm" (FISTA) proposed in [3], which is suited for solving the (possibly strongly) convex problem (6) by means of an appropriate scaled and inexact inertial forward-backward splitting.…”

Section: Sage-fista: Description and Convergence Analysismentioning

confidence: 99%

Scaled, inexact and adaptive generalized FISTA for strongly convex optimization

Rebegoldi¹,

Calatroni²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider a variable metric and inexact version of the FISTA-type algorithm considered in [11,13] for the minimization of the sum of two (possibly strongly) convex functions. The proposed algorithm is combined with an adaptive (non-monotone) backtracking strategy, which allows for the adjustment of the algorithmic step-size along the iterations in order to improve the convergence speed. We prove a linear convergence result for the function values, which depends on both the strong convexity moduli of the two functions and the upper and lower bounds on the spectrum of the variable metric operators. We validate the proposed algorithm, named Scaled Adaptive GEneralized FISTA (SAGE-FISTA), on exemplar image denoising and deblurring problems where edge-preserving Total Variation (TV) regularization is combined with Kullback-Leibler-type fidelity terms, as it is common in applications where signal-dependent Poisson noise is assumed in the data.

show abstract

Recent Advances in Variable Metric First-Order Methods

Cited by 15 publications

References 65 publications

On the inexact scaled gradient projection method

On the inexact scaled gradient projection method

A scaled, inexact and adaptive Fast Iterative Soft-Thresholding Algorithm for convex image restoration

Scaled, inexact and adaptive generalized FISTA for strongly convex optimization

Contact Info

Product

Resources

About