Two-step conjugate gradient method for unconstrained optimization

Abstract

Using Taylor's series, we propose a modified secant relation to get a more accurate approximation of the second curvature of the objective function. Then, using this relation and an approach introduced by Dai and Liao, we present a conjugate gradient algorithm to solve unconstrained optimization problems. The proposed method makes use of both gradient and function values, and utilizes information from the two most recent steps, while the usual secant relation uses only the latest step information. Under appropriate conditions, we show that the proposed method is globally convergent without needing convexity assumption on the objective function. Comparative results show computational efficiency of the proposed method in the sense of the Dolan-Moré performance profiles.

A survey on the Dai–Liao family of nonlinear conjugate gradient methods

“…That is, in [89] an adaptive version of the modified secant equation of [119] has been used. Dehghani and Bidabadi [44] put forward another DL type algorithm using the modified secant equation proposed by Yuan [112]. Inherited from the corresponding modified secant equations, all the modified DL methods suggested in [30,44,70,89,119] benefit the objective function values in addition to the gradient information.…”

“…Dehghani and Bidabadi [44] put forward another DL type algorithm using the modified secant equation proposed by Yuan [112]. Inherited from the corresponding modified secant equations, all the modified DL methods suggested in [30,44,70,89,119] benefit the objective function values in addition to the gradient information. Also, they have been shown to be globally convergent for uniformly convex functions with the modified versions of (1.10), while, for which global convergence regardless of the convexity has been established with the modified versions of (1.11).…”

“…To get global convergence for general functions without any restriction on the CG parameter (1.10), Zhou and Zhang [127] applied the modified secant equation of [67]. In a generalization scheme, to simultaneously take advantage of the objective function values as in [30,44,70,89,119], and to get global convergence without convexity supposition or the mentioned nonnegativity restriction, Arazm et al [13] proposed a modified DL method using the extended secant equation (2.1). Another study with similar aim has been carried out by Dehghani et al [45] with improving the method of [30] based on the modified secant equation of [67].…”