New Two-step Conjugate Gradient Method for Unconstrained Optimization

Abstract

Two-step methods are secant-like techniques of the quasi-Newton type that, unlike the classical methods, construct nonlinear alternatives to the quantities used in the so-called Secant equation. Two-step methods instead incorporate data available from the two most recent iterations and thus create an alternative to the Secant equation with the intention of creating better Hessian approximations that induce faster convergence to the minimizer of the objective function. Such methods, based on reported numerical results published in several research papers related to the subject, have introduced substantial savings in both iteration and function evaluation counts. Encouraged by the successful performance of the methods, we explore in this paper employing them in developing a new Conjugate Gradient (CG) algorithm. CG methods gain popularity on big problems and in situations when memory resources are scarce. The numerical experimentations on the new methods are encouraging and open venue for further investigation of such techniques to explore their merits in a multitude of applications.

On image restoration problems using new conjugate gradient methods

“…They proved to be really effective in practice and showed mature convergence properties. This idea was presented as a technical method in some manuscripts, for example, to maximize in the advantages of the original respective conjugate gradient methods [7], [14]- [16]. To introduce a new method, we will find the Hessian approximation of the minimum of a function f(u), which give a new search direction and choose the coefficient conjugate satisfies above relation.…”

“…For general nonlinear functions, it is necessary to use an iterative procedure [6]. We use the typical Wolfe criteria to determine the step length throughout our operation, as shown in ( 6) and (7):…”

“…𝑑 𝑘 𝑇 𝑔(𝑢 𝑘 + 𝛼 𝑘 𝑑 𝑘 ) ≥ 𝜎 𝑑 𝑘 𝑇 𝑔 𝑘 (7) where 0 < 𝛿 < 𝜎 < 1, Moghrabi [7]. In conjugate gradient methods, the search directions can be defined recursively:…”