Global convergence of the method of shortest residuals

Abstract

The method of shortest residuals (SR) was presented by Hestenes and studied by Pytlak. If the function is quadratic, and if the line search is exact, then the SR method reduces to the linear conjugate gradient method. In this paper, we put forward the formulation of the SR method when the line search is inexact. We prove that, if stepsizes satisfy the strong Wolfe conditions, both the Fletcher-Reeves and Polak-Ribière-Polyak versions of the SR method converge globally. When the Wolfe conditions are used, the two versions are also convergent provided that the stepsizes are uniformly bounded; if the stepsizes are not bounded, an example is constructed to show that they need not converge. Numerical results show that the SR method is a promising alternative of the standard nonlinear conjugate gradient method. Subject Classifications (1991): 65k, 90c Mathematics

Comments on Global convergence of the method of shortest residuals by Y. Dai and Y. Yuan

“…It is claimed in [1] that the procedure stated in [2], which finds α k satisfying (3)-(4) in a finite number of operations, is not correct.…”

Preconditioned conjugate gradient algorithms for nonconvex problems with box constraints

“…In our numerical comparisons we also considered a line search rule based on the Wolfe conditions accommodating the box constraints-that directional minimization rule is an extension of the rule stated in [12]. LSR2: find a positive number α k such that…”

“…has striking resemblance to the Polak-Ribière formula and has not only superior numerical properties but also convergence properties better than that of all existing versions of the Polak-Ribière algorithm (see, e.g., [2], [12], [13], [16], [18] and [29]). In [28] the preconditioned version of the algorithm is introduced.…”

“…This variant of the method of shortest residuals (dealing with non-preconditioned conjugate gradient method) was introduced in [12]. The motivation behind it was to enable the use of Wolfe line search conditions, for which it was necessary to replace the condition (17) by (20)-see [12]. Now consider the problem with box constraints (2).…”

Preconditioned Conjugate Gradient Algorithms for Nonconvex Problems with Box Constraints

“…In [9] (see also [4]) a new family of conjugate gradient algorithms has been introduced whose direction finding subproblem is given by where N r { a , b ) is defined as the point from a line segment spanned by the vectors a and b which has the smallest norm, i.e.,…”