Inexact Restoration approach for minimization with inexact evaluation of the objective function

Abstract

A new method is introduced for minimizing a function that can be computed only inexactly, with different levels of accuracy. The challenge is to evaluate the (potentially very expensive) objective function with low accuracy as far as this does not interfere with the goal of getting high accuracy minimization at the end. For achieving this goal the problem is reformulated in terms of constrained optimization and handled with an Inexact Restoration technique. Convergence is proved and numerical experiments motivated by Electronic Structure Calculations are presented, which indicate that the new method overcomes current approaches for solving large-scale problems.

Inexact Restoration for Minimization with Inexact Evaluation both of the Objective Function and the Constraints

“…By (31), (35), and (40) we have that f , h , and g are bounded above by C f , C h , and C g respectively. Then, as ρ + 1 = 1 θ , we obtain that…”

“…The idea of using the IR framework to deal with optimization problems in which the objective function is subject to evaluation errors comes from [40], where inexactness came from the fact that the evaluation was the result of an iterative process. Evaluating the function with additional precision was considered in [40] as a sort of inexact restoration. This basic principle was developed in [11] and [12], where complexity results were also proved.…”

“…Inexactness of the objective function in optimization problems was addressed in several additional papers in recent years [7,8,9,34,35,39,33]. The objective of the present paper is to use the ideas of [40,11,12] to handle the constrained optimization problem in which the evaluation of the objective functions and the constraints is subject to error. We will show that, although the main ideas are applicable, a number of technical difficulties appear whose solution offer additional insight in the problem.…”

A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients

“…This motivates the derivation of methods that approximate the function and/or the gradient and even the Hessian through a subsampling. This topic has been widely studied recently, see for example [6][7][8]14,20,21,23]. In these papers the data-fitting problem involves a sum, over a large number of measurements, of the misfits.…”

“…In these papers the data-fitting problem involves a sum, over a large number of measurements, of the misfits. In [7,8,20,23] exact and inexact Newton methods and line-search methods based on approximations of the gradient and the Hessian obtained through subsampling are considered, in [21]the problem is reformulated in terms of constrained optimization and handled with an Inexact Restoration technique. In [14] the stochastic gradient method is applied to the approximated problems and conditions on the size of the subproblems are given to maintain the rate of convergence of the full gradient method.…”