Simple Hyper-Heuristics Control the Neighbourhood Size of Randomised Local Search Optimally for LeadingOnes

Abstract

Selection hyper-heuristics (HHs) are randomised search methodologies which choose and execute heuristics during the optimisation process from a set of low-level heuristics. A machine learning mechanism is generally used to decide which low-level heuristic should be applied in each decision step. In this paper we analyse whether sophisticated learning mechanisms are always necessary for HHs to perform well. To this end we consider the most simple HHs from the literature and rigorously analyse their performance for the LEADINGONES benchmark function. Our analysis shows that the standard Simple Random, Permutation, Greedy and Random Gradient HHs show no signs of learning. While the former HHs do not attempt to learn from the past performance of low-level heuristics, the idea behind the Random Gradient HH is to continue to exploit the currently selected heuristic as long as it is successful. Hence, it is embedded with a reinforcement learning mechanism with the shortest possible memory. However, the probability that a promising heuristic is successful in the next step is relatively low when perturbing a reasonable solution to a combinatorial optimisation problem. We generalise the 'simple' Random Gradient HH so success can be measured over a fixed period of time τ , instead of a single iteration. For LEADINGONES we prove that the Generalised Random Gradient (GRG) HH can learn to adapt the neighbourhood size of Randomised Local Search to optimality during the run. As a result, we prove it has the best possible performance achievable with the low-level heuristics (Randomised Local Search with different neighbourhood sizes), up to lower order terms. We also prove that the performance of the HH improves as the number of low-level local search heuristics to choose from increases. In particular, with access to k low-level local search heuristics, it outperforms the best-possible algorithm using any subset of the k heuristics. Finally, we show that the advantages of GRG over Randomised Local Search and Evolutionary Algorithms using standard bit mutation increase if the anytime performance is considered (i.e., the performance gap is larger if approximate solutions are sought rather than exact ones). Experimental analyses confirm these results for different problem sizes (up to n = 10 8 ) and shed some light on the best choices for the parameter τ in various situations. * An extended abstract of this manuscript has appeared at the 2017 Genetic and Evolutionary Computation Conference (GECCO 2017) (Lissovoi et al., 2017).

Maximizing Drift Is Not Optimal for Solving OneMax

“…All these references consider the optimization of ONEMAX, the problem of maximizing the counting-ones function OM : {0, 1} n → R, x → n i=1 x i . Only few theoretical results analyzing algorithms with adaptive parameters consider different functions, e.g., (Lissovoi et al, 2020;Doerr et al, 2018b,a) (see for a complete list of references). ONEMAX also plays a prominent role in empirical research on parameter control.…”

Self-adjusting evolutionary algorithms for multimodal optimization

“…, } for > 1 provably outperform static settings [11] of the mutation operator. Self-adjusting schemes are also closely related to hyper-heuristics which, e. g., can dynamically choose between different mutation operators and therefore outperform static settings [25]. Besides the mutation probability, other parameters like the population sizes may be adjusted during the run of an evolutionary algorithm (EA) and analyzed from a runtime perspective [23].…”

How the Duration of the Learning Period Affects the Performance of Random Gradient Selection Hyper-Heuristics

“…The Generalised Random Gradient (GRG) hyperheuristic was analysed (Lissovoi, Oliveto, and Warwicker 2019b) as an extension of the classical Random Gradient hyper-heuristic (Cowling, Kendall, and Soubeiga 2001). It applies a randomly chosen low-level heuristic for a learning period of τ iterations.…”

“…All rights reserved. multimodal benchmark functions (Alanazi and Lehre 2016;Lehre andÖzcan 2013;Alanazi and Lehre 2014;Qian, Tang, and Zhou 2016;Lissovoi, Oliveto, and Warwicker 2019b;2019a;Doerr et al 2018;Doerr, Doerr, and Yang 2016a). For an overview of theoretical results regarding the performance of hyper-heuristics and other parameter control mechanisms, we refer to the recent survey by Doerr and Doerr (2019).…”

“…It is well understood that random gradient hyperheuristics need to apply low-level heuristics for some time, i.e., the learning period τ , to estimate how good or bad the chosen heuristic is for the current region of the search space (Lissovoi, Oliveto, and Warwicker 2019b). In particular, Lissovoi et al analysed the Generalised Random Gradient (GRG) hyper-heuristic that applies a randomly chosen low-level heuristic for τ iterations and restarts the learning period whenever the fitness value is improved.…”