New convergence results for the scaled gradient projection method

Bonettini, Silvia; Prato, Marco

doi:10.1088/0266-5611/31/9/095008

Cited by 54 publications

(61 citation statements)

References 53 publications

(130 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, which still guarantees the stationarity of the limit points [8]. The aim of this paper is to prove the convergence of this modified SGP scheme if the (nonconvex) objective function Ψ satisfies the Kurdyka-Lojasiewicz (KL) property [15,14], which holds true for most of the functions commonly used in inverse problems as p norms, Kullback-Leibler divergence and indicator functions of box plus equality constraints.…”

Section: The Scaled Gradient Projection Methodsmentioning

confidence: 99%

See 1 more Smart Citation

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

Prato

Bonettini

Loris

et al. 2016

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Abstract. The scaled gradient projection (SGP) method is a first-order optimization method applicable to the constrained minimization of smooth functions and exploiting a scaling matrix multiplying the gradient and a variable steplength parameter to improve the convergence of the scheme. For a general nonconvex function, the limit points of the sequence generated by SGP have been proved to be stationary, while in the convex case and with some restrictions on the choice of the scaling matrix the sequence itself converges to a constrained minimum point. In this paper we extend these convergence results by showing that the SGP sequence converges to a limit point provided that the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain and its gradient is Lipschitz continuous.

show abstract

Section: The Scaled Gradient Projection Methodsmentioning

confidence: 99%

“…[4]. Convergence of the sequence to a minimum point of (1) has recently been proved for convex objective functions by choosing suitable adaptive bounds for the eigenvalues of the scaling matrices [8]. In the following, we will consider a modified version of SGP in which, at each iteration k ∈ N, we compute…”

Section: The Scaled Gradient Projection Methodsmentioning

confidence: 99%

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

Prato

Bonettini

Loris

et al. 2016

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…In general, problem (2) does not have a closed-form solution on account of the inequality constraints, even for simple regularizations, hence an iterative solver must be used. Several resolution approaches are available, either based on projected gradient strategies [39,40], ADMM [41], primal-dual schemes [42], or interior point techniques [43]. Standard interior point methods require to invert several n × n linear systems, which leads to a high computational complexity for large scale problems.…”

Section: Interior Point Approachesmentioning

confidence: 99%

Deep unfolding of a proximal interior point method for image restoration

Bertocchi¹,

Chouzenoux²,

Corbineau³

et al. 2020

Inverse Problems

Self Cite

108

View full text Add to dashboard Cite

Variational methods are widely applied to ill-posed inverse problems for they have the ability to embed prior knowledge about the solution. However, the level of performance of these methods significantly depends on a set of parameters, which can be estimated through computationally expensive and timeconsuming methods. In contrast, deep learning offers very generic and efficient architectures, at the expense of explainability, since it is often used as a black-box, without any fine control over its output. Deep unfolding provides a convenient approach to combine variational-based and deep learning approaches. Starting from a variational formulation for image restoration, we develop iRestNet, a neural network architecture obtained by unfolding a proximal interior point algorithm. Hard constraints, encoding desirable properties for the restored image, are incorporated into the network thanks to a logarithmic barrier, while the barrier parameter, the stepsize, and the penalization weight are learned by the network. We derive explicit expressions for the gradient of the proximity operator for various choices of constraints, which allows training iRestNet with gradient descent and backpropagation. In addition, we provide theoretical results regarding the stability of the network for a common inverse problem example. Numerical experiments on image deblurring problems show that the proposed approach compares favorably with both state-of-the-art variational and machine learning methods in terms of image quality.

show abstract

“…The projected quasi-Newton (PQN) algorithm [65,64] is perhaps the most elegant and logical extension of quasi-Newton methods, but it involves solving a sub-iteration or need to be restricted to a diagonal metric in the implementation [13,12]. PQN proposes the SPG [8] algorithm for the subproblems, and finds that this is an efficient trade-off whenever the cost function (which is not involved in the sub-iteration) is significantly more expensive to evaluate than projecting onto the constraints.…”

Section: Relation To Prior Workmentioning

confidence: 99%

On Quasi-Newton Forward-Backward Splitting: Proximal Calculus and Convergence

Becker¹,

Fadili²,

Ochs³

2019

SIAM J. Optim.

View full text Add to dashboard Cite

We introduce a framework for quasi-Newton forward-backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal ± rank-r symmetric positive definite matrices. This special type of metric allows for a highly efficient evaluation of the proximal mapping. The key to this efficiency is a general proximal calculus in the new metric. By using duality, formulas are derived that relate the proximal mapping in a rank-r modified metric to the original metric. We also describe efficient implementations of the proximity calculation for a large class of functions; the implementations exploit the piece-wise linear nature of the dual problem. Then, we apply these results to acceleration of composite convex minimization problems, which leads to elegant quasi-Newton methods for which we prove convergence. The algorithm is tested on several numerical examples and compared to a comprehensive list of alternatives in the literature. Our quasi-Newton splitting algorithm with the prescribed metric compares favorably against state-of-the-art. The algorithm has extensive applications including signal processing, sparse recovery, machine learning and classification to name a few. Contributions.We introduce an general proximal calculus in a metric V = P ± Q ∈ S ++ (N ) given by P ∈ S ++ (N ) and a positive semi-definite rank-r matrix Q. This significantly extends the result in the preliminary version of this paper [7], where only V = P + Q with a rank-1 matrix Q is addressed. The general calculus is accompanied by several more concrete examples (see Section 3.3.4 for a non-exhaustive list), where, for example, the piecewise linear nature of certain dual problems is rigorously exploited.

show abstract

New convergence results for the scaled gradient projection method

Cited by 54 publications

References 53 publications

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

On the constrained minimization of smooth Kurdyka—Łojasiewicz functions with the scaled gradient projection method

Deep unfolding of a proximal interior point method for image restoration

On Quasi-Newton Forward-Backward Splitting: Proximal Calculus and Convergence

Contact Info

Product

Resources

About