Preconditioned conjugate gradient algorithm for large scale problems with box constraints

Abstract

The paper describes a new conjugate gradient algorithm for large scale nonconvex problems with box constraints. In order to speed up the convergence the algorithm employs a scaling matrix which transforms the space of original variables into the space in which Hessian matrices of functionals describing the problems have more clustered eigenvalues. This is done efficiently by applying limited memory BFGS updating matrices. Once the scaling matrix is calculated, the next few iterations of the conjugate gradient algorithms are performed in the transformed space. The box constraints are treated efficiently by the projection. We believe that the preconditioned conjugate gradient algorithm is competitive to the L-BFGS-B algorithm. We give some numerical results which support our claim.

Preconditioned conjugate gradient algorithms with column scaling

“…R-HPCG Algorithm is the extension of the L-HRH algorithm by giving more flexibility with the inclusion in the formula for d k the scaled previous direction. Furthermore, it is rather straightforward to extend it to problems with box constraints along the lines stated in [11]-the method in [11] outperforms the benchmark code L-BFGS-B ( [17]) on problems from the CUTE collections. The method presented in the paper should inherit superior numerical properties of the algorithm from [11] but at the same time requires less computational effort.…”

“…, g k }. For further discussion crucial is the following result which we present in more general setting 47th IEEE CDC, Cancun, Mexico, Dec. [9][10][11]2008 TuA16.6…”

“…47th IEEE CDC, Cancun, Mexico, Dec. [9][10][11]2008 TuA16.6 Q k is the product of m Givens rotations so the cost of its applying to a given vector is proportional to m. Thus, the evaluatioñ…”

Limited memory quasi-Newton methods for problems with box constraints

“…(25) Proof: The proof of the theorem, as the proofs of other results presented in the paper, is given in [16], or can be derived from the results presented there.…”

“…The second line search rule could be regarded as a variant of R1-R2: find a positive number α k which fulfils (16) and…”