A Comparison of Global Search Algorithms for Continuous Black Box Optimization

Abstract

Four methods for global numerical black box optimization with origins in the mathematical programming community are described and experimentally compared with the state of the art evolutionary method, BIPOP-CMA-ES. The methods chosen for the comparison exhibit various features that are potentially interesting for the evolutionary computation community: systematic sampling of the search space (DIRECT, MCS) possibly combined with a local search method (MCS), or a multi-start approach (NEWUOA, GLOBAL) possibly equipped with a careful selection of points to run a local optimizer from (GLOBAL). The recently proposed “comparing continuous optimizers” (COCO) methodology was adopted as the basis for the comparison. Based on the results, we draw suggestions about which algorithm should be used depending on the available budget of function evaluations, and we propose several possibilities for hybridizing evolutionary algorithms (EAs) with features of the other compared algorithms.

Searching for structural bias in particle swarm optimization and differential evolution algorithms

“…Also, in various practical applications the MNFCs noticeably differ. It has already been shown by Pošik et al (2012) that metaheuristics are mostly useful when the MNFC is relatively large; when it is small, mathematical programming methods are suggested. This is why we conducted our main experiments with a MNFC set to 10,000 and 300,000.…”

“…Although relationships between the types of problems, or their specific features, and the performance of algorithms may be identified Engelbrecht 2013, 2014;Muñoz and Smith-Miles 2016), such relations are very rarely studied in practice. As a result, when metaheuristics are applied to solve practical tasks, despite their popularity and wide acclamation (Fogel 2000;Eiben and Smith 2015;Zelinka 2015), they are often outperformed by problem-specific algorithms (Droste et al 2006), or, when computational budget is limited, mathematical programming approaches (Pošik et al 2012). Moreover, although the convergence of most metaheuristics cannot be proven, one may for some of them prove that on specific types of problems the convergence of such methods cannot be guaranteed (Hu et al 2016).…”

An adaptive multi-spline refinement algorithm in simulation based sailboat trajectory optimization using onboard multi-core computer systems

“…A typical example of this is when the performance measure values are received from computer simulation (see, e.g., Dębski, 2014a). In such an instance, most classic optimization methods cannot be used (at least not directly) and the optimization process is often based on soft-computing/AI methods (Vasile and Locatelli, 2009;Ceriotti and Vasile, 2010;Pošík et al, 2012;Szłapczyński and Szłapczyńska, 2012;Zamuda and Sosa, 2014;Sun and Wu, 2011;Ćurković et al, 2009;Li and Lü, 2014;Bai et al, 2012;Kojic et al, 2013;Zhou et al, 2011).…”

Simulation-Based Sailboat Trajectory Optimization Using on-Board Heterogeneous Computers

“…A typical example of this situation is when the performance measure values are received from computer simulation. In this case, most classic optimization methods cannot be used (at least not directly), and the optimization process is often based on soft-computing methods (see, for instance, [6,27,28,36,38]). …”