The Impact of Noise on Evaluation Complexity: The Deterministic Trust-Region Case

Abstract

Intrinsic noise in objective function and derivatives evaluations may cause premature termination of optimization algorithms. Evaluation complexity bounds taking this situation into account are presented in the framework of a deterministic trust-region method. The results show that the presence of intrinsic noise may dominate these bounds, in contrast with what is known for methods in which the inexactness in function and derivatives' evaluations is fully controllable. Moreover, the new analysis provides estimates of the optimality level achievable, should noise cause early termination. Numerical experiments are reported that support the theory. The analysis finally sheds some light on the impact of inexact computer arithmetic on evaluation complexity.

Asymmetric space-dependent systems: partial stabilization through the addition of noise and exact solutions for the corresponding nonlinear Langevin equations

“…This kind of comparison can be viewed as a heuristic (but not sensibly departing from more rigorous approaches in the literature, e.g. as those in [92,93]) or even an operational way of inferring how stochasticity can lead to the emergence of complexity [94] in certain classes of processes such as [95,96]: dynamics in networks, lasing in noisy media, pattern-formation, granular matter nucleation and ecological interactions, to cite a few examples.…”

A Second-Order Finite-Difference Method for Derivative-Free Optimization

“…Tey also proposed a nonaccelerated derivative-free method similar to the stochastic-gradient-based method and proved an l 1 -norm proximal setup has better complexity bound than the Euclidean proximal setup. Bellavia et al [21] presented evaluation complexity bounds in the framework of a deterministic trust-region method. Tey also showed that the presence of intrinsic noise might dominate the bound and provided estimates of the optimality level achievable, should noise cause early termination.…”