Efficient solvers for Armijo's backtracking problem

Abstract

Backtracking is an inexact line search procedure that selects the first value in a sequence 𝑥 0 , 𝑥 0 𝛽, 𝑥 0 𝛽 2 ... that satisfies 𝑔(𝑥) ≤ 0 on R + with 𝑔(𝑥) ≤ 0 iff 𝑥 ≤ 𝑥 * . This procedure is widely used in descent direction optimization algorithms with Armijo-type conditions. It both returns an estimate in (𝛽𝑥 * , 𝑥 * ] and enjoys an upper-bound ⌈log 𝛽 𝜖/𝑥 0 ⌉ on the number of function evaluations to terminate, with 𝜖 a lower bound on 𝑥 * . The basic bracketing mechanism employed in several root-searching methods is adapted here for the purpose of performing inexact line searches, leading to a new class of inexact line search procedures. The traditional bisection algorithm for root-searching is transposed into a very simple method that completes the same inexact line search in at most ⌈log 2 log 𝛽 𝜖/𝑥 0 ⌉ function evaluations. A recent bracketing algorithm for root-searching which presents both minmax function evaluation cost (as the bisection algorithm) and superlinear convergence is also transposed, asymptotically requiring ∼ log log log 𝜖/𝑥 0 function evaluations for sufficiently smooth functions. Other bracketing algorithms for root-searching can be adapted in the same way. Numerical experiments suggest time savings of 50% to 80% in each call to the inexact search procedure.CCS Concepts: • Mathematics of computing → Solvers; Numerical analysis.