Efficient Multi-start Strategies for Local Search Algorithms

Abstract

Abstract. Local search algorithms for global optimization often suffer from getting trapped in a local optimum. The common solution for this problem is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resources (in particular, processing time) to the instances depending on their behavior. Hence, a multi-start strategy has to decide (dynamically) when to allocate additional resources to a particular instance and when to start new instances. In this paper we propose a consistent multi-start strategy that assumes a convergence rate of the local search algorithm up to an unknown constant, and in every phase gives preference to those instances that could converge to the best value for a particular range of the constant. Combined with the local search algorithm SPSA (Simultaneous Perturbation Stochastic Approximation), the strategy performs remarkably well in practice, both on synthetic tasks and on tuning the parameters of learning algorithms.

Assessment of Modeling Uncertainties Using a Multistart Optimization Tool for Surface Complexation Equilibrium Parameters (MUSE)

“…However, similar to other minimization techniques, Nelder–Mead is dependent on the choice of starting points and boundary conditions and is often trapped in a local minimum around the initial guesses. To address this problem, the Nelder–Mead method was hybridized with a multistart algorithm, an approach that has shown promise in other fields for global optimization problems. − …”

Multi criteria biased randomized method for resource allocation in distributed systems: Application in a volunteer computing system

“…Therefore, we have defined a method to obtain it. We propose a Multi-Start procedure [27] which executes several instances of the heuristic depicted in Algorithm 3, and chooses the best one as initial solution. This heuristic works as follows.…”