A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables

Abstract

We propose a gradient-based method for quadratic programming problems with a single linear constraint and bounds on the variables. Inspired by the GPCG algorithm for boundconstrained convex quadratic programming [J.J. Moré and G. Toraldo, SIAM J. Optim. 1, 1991], our approach alternates between two phases until convergence: an identification phase, which performs gradient projection iterations until either a candidate active set is identified or no reasonable progress is made, and an unconstrained minimization phase, which reduces the objective function in a suitable space defined by the identification phase, by applying either the conjugate gradient method or a recently proposed spectral gradient method. However, the algorithm differs from GPCG not only because it deals with a more general class of problems, but mainly for the way it stops the minimization phase. This is based on a comparison between a measure of optimality in the reduced space and a measure of bindingness of the variables that are on the bounds, defined by extending the concept of proportional iterate, which was proposed by some authors for box-constrained problems. If the objective function is bounded, the algorithm converges to a stationary point thanks to a suitable application of the gradient projection method in the identification phase. For strictly convex problems, the algorithm converges to the optimal solution in a finite number of steps even in case of degeneracy. Extensive numerical experiments show the effectiveness of the proposed approach.Gradient Projection (GP) methods are widely used to solve large-scale SLBQP problems, thanks to the availability of low-cost projection algorithms onto the feasible set of (1.1) (see, e.g, [8,12,9]). In particular, Spectral Projected Gradient methods [4], other GP methods exploiting variants of Barzilai-Borwein (BB) steps [35,12], and more recent Scaled Gradient Projection methods [6] have proved their effectiveness in several applications.Bound-constrained Quadratic Programming problems (BQPs) can be regarded as a special case of SLBQPs, where the theory or the implementation can be simplified. This has favoured the development of more specialized gradient-based methods, built upon the idea of combining steps aimed at identifying the variables that are active at a solution (or at a stationary point) with unconstrained minimizations in reduced spaces defined by fixing the variables that are estimated active [31,24,32,25,3,20,22,27,21,28]. Thanks to the identification properties of the GP method [7] and to its capability of adding/removing multiple variables to/from the active set in a single iteration, GP steps are a natural choice to determine the active variables. A wellknown method based on this approach is GPCG [32], developed for strictly convex BQPs. It alternates between two phases: an identification phase, which performs GP iterations until a suitable face of the feasible set is identified or no reasonable progress toward the solution is achieved, and a minimization phase, which uses the ...

Steplength selection in gradient projection methods for box-constrained quadratic programs

“…In order to verify the above conclusions on a more exhaustive set of large-scale problems, we used the software package downloadable at http://www.dimat.unina2.it/diserafino/dds sw.htm for randomly generating box-constrained quadratic problems. Following the procedure proposed in [32,33], the software allows to generate test problems with different size, spectral properties and number of active constraints at the solution. We generated a dataset of 108 strictly convex QP problems with nondegenerate solutions, splitted into three groups of increasing size: n = 15000, 20000, 25000; for each group, 36 problems are generated by considering three values for the number na of active constraints at the solution, na ≈ 0.1 • n, 0.5 • n, 0.9 • n, three values for the condition number κ(A) of the Hessian matrix A, κ(A) = 10 4 , 10 5 , 10 6 , and four levels of near-degeneracy, obtained by setting the positive Lagrangian multipliers β * i , i = 1, .…”

“…where ϕ(x (k) ) is the vector with the entries defined in (33). As in [34], given a set P of n p problems, for each problem p we consider the ratio r p,s of the computing time of a solver s versus the best time of all the solvers (performance ratio) and, for each solver s and for θ ≥ 1 define…”

ACQUIRE: an inexact iteratively reweighted norm approach for TV-based Poisson image restoration

“…The second condition in (12) can be achieved by using any constrained minimization algorithm. We note that, for the restoration problems considered in this work, gradient-projection methods, such as those in [10,23,37], are suited to the solution of the inner problems (11). Indeed, numerical experiments have shown that very low accuracy is required in practice in the solution of the inner problems; furthermore, the computational cost per iteration of gradient projection methods is modest when low-cost algorithms for the projection onto the feasible set are available.…”

Sparsity-Aware Noise Subspace Fitting for DOA Estimation

“…The optimization problem in Equation (17) is an LCQP problem that can be solved by off-the-shelf toolbox [35,36,40]. Here, we employ the MATLAB's quadprog solver to implement SANSF, in which the computational complexity is on the order of O(8P 3 ) real operations [41].…”