On the regularizing behavior of the SDA and SDC gradient methods in the solution of linear ill-posed problems

Asmundis, Roberta De; Serafino, Daniela di; Landi, Germana

doi:10.1016/j.cam.2016.01.007

Cited by 19 publications

(10 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…is the Yuan steplength [37]. The interest for this steplength is motivated by its spectral properties, which dramatically speed up the convergence [14,18], while showing certain regularization properties useful to deal with linear ill-posed problems [15]. Similar properties hold for the SDA gradient method [16], but for the sake of space we do not show the results of its application in the minimization phase.…”

Section: Identification Phasementioning

confidence: 99%

“…In the minimization phase, we use the CG method, and, in the strictly convex case, we also use the SDC method proposed in [14]. This provides a way to extend SDC to the costrained case, with the goal of exploiting its smoothing and regularizing effect observed on certain unconstrained ill-posed inverse problems [15]. Of course, the CG solver is still the reference choice in general, especially because it is able to deal with nonconvexity through directions of negative curvature (as done, e.g., in [30]), whereas handling negative curvatures with spectral gradient methods may be a non-trivial task (see, e.g., [10] and the references therein).…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables

Serafino¹,

Toraldo²,

Viola³

et al. 2018

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

We propose a gradient-based method for quadratic programming problems with a single linear constraint and bounds on the variables. Inspired by the GPCG algorithm for boundconstrained convex quadratic programming [J.J. Moré and G. Toraldo, SIAM J. Optim. 1, 1991], our approach alternates between two phases until convergence: an identification phase, which performs gradient projection iterations until either a candidate active set is identified or no reasonable progress is made, and an unconstrained minimization phase, which reduces the objective function in a suitable space defined by the identification phase, by applying either the conjugate gradient method or a recently proposed spectral gradient method. However, the algorithm differs from GPCG not only because it deals with a more general class of problems, but mainly for the way it stops the minimization phase. This is based on a comparison between a measure of optimality in the reduced space and a measure of bindingness of the variables that are on the bounds, defined by extending the concept of proportional iterate, which was proposed by some authors for box-constrained problems. If the objective function is bounded, the algorithm converges to a stationary point thanks to a suitable application of the gradient projection method in the identification phase. For strictly convex problems, the algorithm converges to the optimal solution in a finite number of steps even in case of degeneracy. Extensive numerical experiments show the effectiveness of the proposed approach.Gradient Projection (GP) methods are widely used to solve large-scale SLBQP problems, thanks to the availability of low-cost projection algorithms onto the feasible set of (1.1) (see, e.g, [8,12,9]). In particular, Spectral Projected Gradient methods [4], other GP methods exploiting variants of Barzilai-Borwein (BB) steps [35,12], and more recent Scaled Gradient Projection methods [6] have proved their effectiveness in several applications.Bound-constrained Quadratic Programming problems (BQPs) can be regarded as a special case of SLBQPs, where the theory or the implementation can be simplified. This has favoured the development of more specialized gradient-based methods, built upon the idea of combining steps aimed at identifying the variables that are active at a solution (or at a stationary point) with unconstrained minimizations in reduced spaces defined by fixing the variables that are estimated active [31,24,32,25,3,20,22,27,21,28]. Thanks to the identification properties of the GP method [7] and to its capability of adding/removing multiple variables to/from the active set in a single iteration, GP steps are a natural choice to determine the active variables. A wellknown method based on this approach is GPCG [32], developed for strictly convex BQPs. It alternates between two phases: an identification phase, which performs GP iterations until a suitable face of the feasible set is identified or no reasonable progress toward the solution is achieved, and a minimization phase, which uses the ...

show abstract

Section: Identification Phasementioning

confidence: 99%

mentioning

confidence: 99%

A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables

Serafino¹,

Toraldo²,

Viola³

et al. 2018

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many real life applications lead to nonlinear optimization problems whose very large size makes first-order methods the most suitable choice. Among first-order approaches, gradient methods have widely proved their effectiveness in solving challenging unconstrained and constrained problems arising in signal and image processing, compressive sensing, machine learning, optics, chemistry and other areas (see, e.g., [1,2,3,4,5,6,7,8,9,10,11] and the references therein).…”

Section: Introductionmentioning

confidence: 99%

On the steplength selection in gradient methods for unconstrained optimization

Serafino

Ruggiero

Toraldo

et al. 2018

Applied Mathematics and Computation

Self Cite

102

View full text Add to dashboard Cite

The seminal paper by Barzilai and Borwein (1988) has given rise to an extensive investigation, leading to the development of effective gradient methods. Several steplength rules have been first designed for unconstrained quadratic problems and then extended to general nonlinear optimization problems. These rules share the common idea of attempting to capture, in an inexpensive way, some second-order information. However, the convergence theory of the gradient methods using the previous rules does not explain their effectiveness, and a full understanding of their practical behaviour is still missing. In this work we investigate the relationships between the steplengths of a variety of gradient methods and the spectrum of the Hessian of the objective function, providing insight into the computational effectiveness of the methods, for both quadratic and general unconstrained optimization problems. Our study also identifies basic principles for designing effective gradient methods

show abstract

“…Mathematical Problems in Engineering gradient descent-based algorithms have been widely adopted in image processing tasks [18,[32][33][34][35][36][37]. As to TV minimization, gradient projection based PDE methods [1] originally were adopted to solve the associated nonlinear Euler-Lagrange equation.…”

Section: Introductionmentioning

confidence: 99%

A Coordinate Descent Method for Total Variation Minimization

Deng

Ren

Xiao³

et al. 2017

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Total variation (TV) is a well-known image model with extensive applications in various images and vision tasks, for example, denoising, deblurring, superresolution, inpainting, and compressed sensing. In this paper, we systematically study the coordinate descent (CoD) method for solving general total variation (TV) minimization problems. Based on multidirectional gradients representation, the proposed CoD method provides a unified solution for both anisotropic and isotropic TV-based denoising (CoDenoise). With sequential sweeping and small random perturbations, CoDenoise is efficient in denoising and empirically converges to optimal solution. Moreover, CoDenoise also delivers new perspective on understanding recursive weighted median filtering. By incorporating with the Augmented Lagrangian Method (ALM), CoD was further extended to TV-based image deblurring (ALMCD). The results on denoising and deblurring validate the efficiency and effectiveness of the CoD-based methods.

show abstract

On the regularizing behavior of the SDA and SDC gradient methods in the solution of linear ill-posed problems

Cited by 19 publications

References 39 publications

A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables

A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables

On the steplength selection in gradient methods for unconstrained optimization

A Coordinate Descent Method for Total Variation Minimization

Contact Info

Product

Resources

About