When move acceptance selection hyper-heuristics outperform Metropolis and elitist evolutionary algorithms and when not

How Well Does the Metropolis Algorithm Cope With Local Optima?

“…Outside the range of well-established search heuristics, Paixão et al [53] show that the strong-selection weak-mutation process from biology can optimize some functions faster than elitist evolutionary algorithms. Lissovoi et al [48] show that the moveacceptance hyper-heuristic proposed by Lehre and Özcan [44] can optimize cliff functions in cubic time. However, as recently shown in [13], it performs significantly worse than most EAs on the Jump benchmark.…”

“…While it is not surprising that the MA does not profit from its ability to accept inferior solutions on this unimodal benchmark, their result shows that only very small temperatures (namely such that the probability of accepting an inferior solution is at most 𝑂 (log(𝑛)/𝑛)) lead to polynomial runtimes. As a side result to their study on hyperheuristics, Lissovoi et al [48,Theorem 14] show that the MA cannot optimize the multimodal Jump benchmark in sub-exponential time. The same work also contains a runtime analysis on the Cliff problem, which we will discuss in more detail after having introduced this benchmark further below.…”

“…We note that the original cliff benchmark is the special case with 𝑑 = 𝑚 − 3/2. To the best of our knowledge, the only runtime analysis of the MA on Cliff functions was conducted by [48,Theorem 11]. On the original Cliff 𝑑,𝑚 function with fixed 𝑑 = 𝑚 − 3/2, the authors showed a lower bound for the runtime of min 1 2…”

“…We now analyze the runtime of the Metropolis on the generalize class of Cliff functions proposed in this work. Compared to the previous analysis on classic Cliff functions in [48], we overcome two additional technical obstacles. The first, obvious one, is that our class of Cliff functions comes with two parameters 𝑚 and 𝑑 and the MA has the parameter 𝛼.…”

“…In this three-dimensional parameter space, complex interactions appear which make the runtime analysis challenging and the final results complex. The second is that we aim at tighter bounds than those shown in [48] and at bounds that are informative also in the super-polynomial regime. For this reason, we cannot assume that 𝛼 = Ω(𝑛/log 𝑛) as otherwise the hill-climbing part would be super-polynomial.…”