Nonlinear Conjugate Gradient Methods

Abstract

Conjugate gradient methods are a class of important methods for solving linear equations and for solving nonlinear optimization. In this article, a review on conjugate gradient methods for unconstrained optimization is given. They are divided into early conjugate gradient methods, descent conjugate gradient methods, and sufficient descent conjugate gradient methods. Two general convergence theorems are provided for the conjugate gradient method assuming the descent property of each search direction. Some research issues on conjugate gradient methods are mentioned.

A new spectral Polak-Ribière-Polyak conjugate gradient method

“…This feature means that the PRP method has been one of the most efficient conjugate gradient methods in practical computation for many years. However, Dai 4 constructed an example to indicate that the PRP method may generate an upward direction resulting in the iterative scheme failing even if the objective function is uniformly convex under the strong Wolfe line search. So far, the convergence of the PRP method has not been completely proved under the Wolfe-type line search.…”

A Globally and R-Linearly Convergent Hybrid HS and PRP Method and its Inexact Version with Applications

“…Dai and Yuan [7] proved that the DY method is a descent method and globally convergent in the case of the standard Wolfe line search. However, the HS method and the PRP method may generate ascent directions even with the strong Wolfe line search [4], which prevent them from global convergence although both methods are regarded as two of the most efficient conjugate gradient methods in practical computation. To guarantee global convergence of the PRP method, some line searches which force it generate descent direction were proposed [4,14].…”

“…However, the HS method and the PRP method may generate ascent directions even with the strong Wolfe line search [4], which prevent them from global convergence although both methods are regarded as two of the most efficient conjugate gradient methods in practical computation. To guarantee global convergence of the PRP method, some line searches which force it generate descent direction were proposed [4,14]. Recently, by the use of an approximate descent backtracking line search, Zhou [25] showed that the original PRP method converges globally even for nonconvex functions whether the search direction is descent or not.…”

“…To improve practical computation efficiency and convergence properties of conjugate gradient methods, many hybrid methods have been proposed, please see the recent survey [4] and references therein. These hybrid methods can be divided into two classes, one is the hybrid FR and PRP type methods such as the hybrid method [13] where…”

A Descent Dai-Liao Conjugate Gradient Method Based on a Modified Secant Equation and Its Global Convergence

“…If f is a strictly convex quadratic function, and the performed line search is exact, all these methods are equivalent, but for a general function different choices of β k give rise to distinct conjugate gradient methods with quite different computational efficiency and convergence properties. We refer to the books [7,8], the survey paper [9], and the references therein about the numerical performance and the convergence properties of conjugate gradient methods. During the last decade, much effort has been devoted to develop new conjugate gradient methods which are not only globally convergent for general functions but also computationally superior to classical methods and there are classified by two classes.…”