Spectral Gradient Methods for Linearly Constrained Optimization

Abstract

Linearly constrained optimization problems with simple bounds are considered in the present work. First, a preconditioned spectral gradient method is defined for the case in which no simple bounds are present. This algorithm can be viewed as a quasiNewton method in which the approximate Hessians satisfy a weak secant equation. The spectral choice of steplength is embedded into the Hessian approximation, and the whole process is combined with a nonmonotone line search strategy. The simple bounds are then taken into account by placing them in an exponential penalty term that modifies the objective function. The exponential penalty scheme defines the outer iterations of the process. Each outer iteration involves the application of the previously defined preconditioned spectral gradient method for linear equality constrained problems. Therefore, an equality constrained convex quadratic programming problem needs to be solved at every inner iteration. The associated extended KKT matrix remains constant unless the process is reinitiated. In ordinary inner iterations, only the right hand side of the KKT system changes. Therefore, suitable sparse factorization techniques can be effectively applied and exploited. Encouraging numerical experiments are presented.

Spectral Projected Gradient Methods: Review and Perspectives

“…The case in which the convex feasible set of the problem to be solved by SPG is defined by linear equality and inequality constraints has been considered in Birgin et al (2003b), Andreani et al (2005), Martínez, Pilotta, and Raydan (2005), and . A crucial observation is that this type of set is not necessarily one in which it is easy to project.…”

Spectral Projected Gradient Methods

“…In [38] the SPG method was used to solve Augmented Lagrangian subproblems. The spectral gradient method solves the subproblems originated by the application of an exponential penalty method to linearly constrained optimization in [56].…”

A new self scaling PCG algorithm with dynamical retards

“…assume that (0,1)   is given independently of the iteration number k and that k d has been computed . and goto step 2 for more detail see [10].…”