On Accelerated Random Search

Abstract

A new variant of pure random search (PRS) for function optimization is introduced. The basic finite-descent accelerated random search (ARS) algorithm is simple: the search is confined to shrinking neighborhoods of a previous record-generating value, with the search neighborhood reinitialized to the entire space when a new record is found. Local maxima are avoided by including an automatic restart feature which reinitializes the search neighborhood after some number of shrink steps have been performed.One goal of this article is to provide rigorous mathematical comparisons of ARS to PRS. It is shown that the sequence produced by the ARS process converges, with probability one, to the maximum of a continuous objective function faster than that of the PRS process by adjustably large multiples of the time step (Theorem 1). Regarding an infinite-descent (no automatic restart) version of ARS, it is shown that if the objective function satisfies a local nonflatness condition, then the right tails of the distributions of inter-record times are exponentially smaller than those of PRS (Theorem 3).Performance comparisons between ARS, PRS, and three quasi-Newton-type optimization routines are reported in attempting to find extrema of (i) each of a small collection of standard test functions of two variables, and (ii) d-dimensional polynomials with random roots. Also reported is a three-way performance comparison between ARS, PRS, and a simulated annealing algorithm in attempting to solve traveling salesman problems.

On the convergence rate issues of general Markov search for global minimum

“…This method was analysed in [3] and it was shown that this algorithm outperforms the simple PRS. As follows from below the finite descent version of ARS (the case ρ > 0) converges lazily towards A .…”

“…This paper shows when an optimization method is lazy and the presented general results cover, in particular, the class of simulated annealing algorithms and monotone random search. To provide an application example from the class of methods which are based on parameters' self-adaptation it is shown that the finite descent Accelerated Random Search [3], converges lazily. As it is discussed further, the undesired lazy convergence property appears to be the property of an optimization method rather than the property of the problem function f .…”

Application of Improved Accelerated Random Search Algorithm for Structural Damage Detection

“…Marwala [34][35][36] successfully applied three separate genetic algorithms to minimize the distance between the measured data and the finite-element predicted data. Touat et al 37 proposed an accelerated random search algorithm for model updating. The Accelerated Random search (ARS) algorithm 38 has been developed to accelerate the stochastic search process for mathematical optimisation problems.…”

Stochastic filter methods for generally constrained global optimization

“…Accelerated random search (ars) [3], and its offshoot oscars [14], are methods for bound constrained global optimization which use the objective function only to determine which of a pair of points is better than the other. Hence they can easily be modified to address other problems such as (1) when an alternative way of doing pairwise comparisons (e.g.…”