A Limited Memory Gradient Projection Method for Box-Constrained Quadratic Optimization Problems

Hybrid limited memory gradient projection methods for box-constrained optimization problems

“…In particular, we study if the LM idea can be borrowed in order to generate m approximations of the eigenvalues of the objective function Hessian matrix restricted to those indices corresponding to the inactive components at the solution. A first attempt to achieve this target has been carried out in [28,30] for GP methods equipped with a line search along the feasible direction. In the following we propose a detailed analysis for the GP method with line search along the projection arc, that deserves an ad hoc study due to its intrinsic differences with respect to the case in which the line search along the feasible direction is used.…”

“…In general, we can not ensure that for m + 1 successive iterations the sets F k+j , j = 0, 1, … , m are all the same and hence, the matrix T F (k+1,k+m) , whose eigenvalues are approximations of those of A F (k+1,k+m) ,F (k+1,k+m) , can not be simply written as in (30). Indeed, in view of (27), the matrix T F (k+1,k+m) has the following form where In practice, the computation of the matrix E F (k+1,k+m) is not affordable and, as a consequence, along the iterative procedure, it is only possible to approximate the eigenvalues of T F (k+1,k+m) and we now investigate how to achieve this goal.…”

“…If instead, F k = ⋯ = F k+m and the corresponding B k = ⋯ = B k+m are consistent, then a LM steplength selection rule can be employed to select the steplength in the next (at most) m iterations. Indeed in this case the original LM steplength rule proposed in [18] can be recovered with respect to the indices in the set F (k+1,k+m) (as explained in Section 3) and the eigenvalues of the matrix T F (k+1,k+m) defined in (30) approximate m eigenvalues of the Hessian submatrix restricted to the intersection of rows and columns with indices in F (k+1,k+m) .…”

“…To reach this goal we continue the analysis carried out in [25] by taking into account more sophisticated steplength techniques. By deepening the results in [28,30], we investigate how to employ, in gradient projection methods for box-constrained minimization problems, the so-called limited memory steplength procedures, based on the storage of a prefixed number of past gradients, such as the ones proposed for unconstrained optimization in [18,21]. In particular, we propose a hybrid steplength selection rule that automatically combines a proper alternation of the modified BB strategies suggested in [25], adopted when the final active set is far to be identified, and the limited memory technique, employed when the sequence generated by the algorithm (2)-(3) provides the same set of active constraints for a suitable number of consecutive iterations.…”