Strong Evaluation Complexity of An Inexact Trust-Region Algorithm for Arbitrary-Order Unconstrained Nonconvex Optimization

Abstract

A trust-region algorithm using inexact function and derivatives values is introduced for solving unconstrained smooth optimization problems. This algorithm uses high-order Taylor models and allows the search of strong approximate minimizers of arbitrary order. The evaluation complexity of finding a q-th approximate minimizer using this algorithm is then shown, under standard conditions, to be O min j∈{1,...,q} ǫ −(q+1) jwhere the ǫ j are the order-dependent requested accuracy thresholds. Remarkably, this order is identical to that of classical trust-region methods using exact information.

Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques

“…of finding approximate minimizers of arbitrary order. Exploiting ideas of [12,7], we have shown that its evaluation complexity is (in expectation) of the same order in the requested accuracy as that known for related deterministic methods [7,10].…”

“…We have considered a trust-region method for unconstrained minimization inspired by [10] which is adapted to handle randomly perturbed function and derivatives values and is capable Appendix: additional proofs Proof of (3.12) Proof. Since σ(½ Λ c k ) belong to A k−1 , because the random variable Λ k is fully determined, assuming P r(½ Λ c k ) > 0, the tower property yields:…”

“…(a vector d solving the optimization problem defining φ δ f,j (x) in (2.5) is called an optimality displacement) [8,10]. In other words, a q-th order (ǫ, δ)-approximate minimizer is a point from which no significant decrease of the Taylor expansions of degree one to q can be obtained in a ball of optimality radius δ.…”

“…Our objective is twofold. Firstly, we introduce a version of the deterministic method proposed in [10] which is able to handle the random context and provide, under reasonable probabilistic assumptions, a sharp evaluation complexity bound (in expectation) for arbitrary optimality order. Secondly, we investigate the effect of intrinsic noise (that is noise whose level cannot be assumed to vanish) on a first-order version of our algorithm and prove "degraded" optimality, should this noise limit the validity of our assumptions.…”

The Impact of Noise on Evaluation Complexity: The Deterministic Trust-Region Case

“…This paper attempts to answer a simple question: how does noise in function values and derivatives affect evaluation complexity of smooth optimization? While analysis has been produced to indicate how high accuracy can be reached by optimization algorithms even in the presence of inexact but deterministic (1) function and derivatives' values (see [8,16,28,3,29,21,14]), these approaches crucially rely on the assumption that the inexactness is controllable, in that it can be made arbitrarily small if required so by the algorithm. But what happens in practical applications where significant noise is intrinsic and can't be assumed away?…”