Curvilinear Stabilization Techniques for Truncated Newton Methods in Large Scale Unconstrained Optimization

Abstract

The aim of this paper is to define a new class of minimization algorithms for solving large scale unconstrained problems. In particular we describe a stabilization framework, based on a curvilinear linesearch, which uses a combination of a Newton-type direction and a negative curvature direction. The motivation for using negative curvature direction is that of taking into account local nonconvexity of the objective function. On the basis of this framework, we propose an algorithm which uses the Lanczos method for determining at each iteration both a Newton-type direction and an effective negative curvature direction. The results of extensive numerical testing are reported together with a comparison with the LANCELOT package. These results show that the algorithm is very competitive, which seems to indicate that the proposed approach is promising.

A curvilinear method based on minimal-memory BFGS updates

“…In both cases, the computation of the negative curvature direction requires the factorization and the storage of a matrix, which is computationally expensive when the number of variables is large. In [11] the computation of the negative curvature direction was based on a Lanczos procedure. However, the storage of a matrix is required and only a few Lanczos vectors are stored.…”

A dwindling filter line search method for unconstrained optimization

“…In [6,21,31,37,38], this approach was embedded in a nonmonotonic framework and used to solving small and large-scale optimization problems. The scaling of two directions s k and d k are taken into account in [13].…”

A Controlled Random Search Algorithm with Local Newton-type Search for Global Optimization

“…Furthermore, as is known, the use of gradient-related directions ensures that any global minimizer has a region of attraction and hence, starting the local search from a point in this region, we can insert quickly a global minimizer in the set of "promising" points. As a local minimization procedure, we use the Newton-type method proposed in [7] which guarantees the convergence to second order critical points, i.e. to stationary points where the Hessian matrix is positive semidefinite.…”

“…In particular we propose an algorithm which combines a controlled random search procedure based on the modified Price algorithm described in [2] with a Newton-type unconstrained minimization algorithm proposed in [7]. More in particular, we exploit the skill of the Price strategy to examine the whole region of interest in order to locate the subregions "more promising" to contain a global minimizer.…”

“…We refer to [7,8] for a detailed analysis of this local algorithm. We just mention here that the main feature of the method is to ensure global convergence towards points satisfying the second order necessary optimality condition, namely stationary points where the Hessian matrix is positive semidefinite.…”