Modification of Nonlinear Conjugate Gradient Method with Weak Wolfe-Powell Line Search

The conjugate gradient (CG) method is a useful tool for obtaining the optimum point for unconstrained optimization problems since it does not require a second derivative or its approximations. Moreover, the conjugate gradient method can be applied in many fields such as machine learning, deep learning, neural network, and many others. This paper constructs a four-term conjugate gradient method that satisfies the descent property and convergence properties to obtain the stationary point. The new modification was constructed based on Liu and Storey's conjugate gradient method, two-term conjugate gradient method, and three-term conjugate gradient method. To analyze the efficiency and robustness, we used more than 150 optimization functions from the CUTEst library with different dimensions and shapes. The numerical results show that the new modification outperforms the recent conjugate gradient methods such as CG-Descent, Dai and Liao, and others in terms of number of functions evaluations, number of gradient evaluations, number of iterations, and CPU time.

show abstract

“…In addition, Hager and Zhang [17,18] presented the following CG formula based on equation (6) given by…”

Section: B -Inexact Line Searchmentioning

confidence: 99%

A Descent Four-Term of Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

Alhawarat

Alolaiyan²,

Masmali

et al. 2021

Eur. J. Pure Appl. Math.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this section, we present six-hump camel back function, which is a multimodal function to test the efficiency of the optimization algorithm. e function is defined as follows: 3 for a three-dimensional graph); for more information about two-dimensional functions, the reader can refer to [19].…”

Section: Multimodal Function With Its Graphmentioning

confidence: 99%

An Efficient Modified AZPRP Conjugate Gradient Method for Large-Scale Unconstrained Optimization Problem

Alhawarat

Nguyen

Sabra

et al. 2021

Journal of Mathematics

Self Cite

View full text Add to dashboard Cite

To find a solution of unconstrained optimization problems, we normally use a conjugate gradient (CG) method since it does not cost memory or storage of second derivative like Newton’s method or Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. Recently, a new modification of Polak and Ribiere method was proposed with new restart condition to give a so-call AZPRP method. In this paper, we propose a new modification of AZPRP CG method to solve large-scale unconstrained optimization problems based on a modification of restart condition. The new parameter satisfies the descent property and the global convergence analysis with the strong Wolfe-Powell line search. The numerical results prove that the new CG method is strongly aggressive compared with CG_Descent method. The comparisons are made under a set of more than 140 standard functions from the CUTEst library. The comparison includes number of iterations and CPU time.

show abstract

“…This restriction motivates us to the construct a new version of the CG method, which is simple and relatively easy to understand. For more information the reader can read the following papers Alhawarat and Salleh (2017), Alhawarat et al (2015), Hestenes and Stiefel (1952), Gilbert and Nocedal (1992) and Salleh and Alhawarat (2016).…”

Section:    mentioning

confidence: 99%

Conjugate Gradient ‎Method: A Developed Version to Resolve Unconstrained Optimization Problems

Alhawarat

Nguyen-Thoi

Salleh

2020

Journal of Computer Science

Self Cite

View full text Add to dashboard Cite

One of the important methods that are widely utilized to resolve unconstrained optimization problems is the Conjugate Gradient (CG) method. This paper aims to propose a new version of the CG method on the basis of Weak Wolfe-Powell (WWP) line search. The assumption is bounded below optimization problems with the Lipschitz continuous gradient. The new parameter obtains global convergence properties when the WWP line search is used. The descent condition is established without using any line search. The performance of the proposed CG method is tested by obtaining some unconstrained optimization problems from the CUTEst library. Testing results show that the proposed version of the CG method outperforms CG-DESCENT version 5.3 in terms of CPU time, function evaluations, gradient evaluations and number of iterations.

show abstract

Modification of Nonlinear Conjugate Gradient Method with Weak Wolfe-Powell Line Search

Cited by 10 publications

References 23 publications

A Descent Four-Term of Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

A Descent Four-Term of Liu and Storey Conjugate Gradient Method for Large Scale Unconstrained Optimization Problems

An Efficient Modified AZPRP Conjugate Gradient Method for Large-Scale Unconstrained Optimization Problem

Conjugate Gradient ‎Method: A Developed Version to Resolve Unconstrained Optimization Problems

Contact Info

Product

Resources

About