Adaptive landscape analysis

Janković, Anja; Doerr, Benjamin

doi:10.1145/3319619.3326905

Cited by 24 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have analyzed the potential impact of such an online selection in [44]. Some initial work in determining how landscape features change during the search has been proposed in [21], but it was shown in [36] that some of the local features provided by flacco are not very robust, so that a suitable selection of features is needed for the use in a landscape-aware algorithm design.…”

Section: Discussionmentioning

confidence: 99%

Integrated vs. sequential approaches for selecting and tuning CMA-ES variants

Vermetten¹,

Wang

Doerr

et al. 2020

Proceedings of the 2020 Genetic and Evolutionary Computation Conference

Self Cite

View full text Add to dashboard Cite

When faced with a specific optimization problem, deciding which algorithm to apply is always a difficult task. Not only is there a vast variety of algorithms to select from, but these algorithms are often controlled by many hyperparameters, which need to be suitably tuned in order to achieve peak performance. Usually, the problem of selecting and configuring the optimization algorithm is addressed sequentially, by first selecting a suitable algorithm and then tuning it for the application at hand. Integrated approaches, commonly known as Combined Algorithm Selection and Hyperparameter (CASH) solvers, have shown promise in several applications. In this work we compare sequential and integrated approaches for selecting and tuning the best out of the 4,608 variants of the modular Covariance Matrix Adaptation Evolution Strategy (CMA-ES). We show that the ranking of these variants depends to a large extent on the quality of the hyperparameters. Sequential approaches are therefore likely to recommend sub-optimal choices. Integrated approaches, in contrast, manage to provide competitive results at much smaller computational cost. We also highlight important differences in the search behavior of two CASH approaches, which build on racing (irace) and on model-based optimization (MIP-EGO), respectively. CCS CONCEPTS • Theory of computation → Evolutionary algorithms; Bioinspired optimization; Algorithm design techniques.

show abstract

Section: Discussionmentioning

confidence: 99%

Integrated vs. sequential approaches for selecting and tuning CMA-ES variants

Vermetten¹,

Wang

Doerr

et al. 2020

Proceedings of the 2020 Genetic and Evolutionary Computation Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…We have analyzed the potential impact of such an online selection in [41]. Some initial work in determining how landscape features change during the search has been proposed in [21], but it was shown in [34] that some of the local features provided by flacco are not very robust, so that a suitable selection of features is needed for the use in a landscape-aware algorithm design.…”

Section: Discussionmentioning

confidence: 99%

Sequential vs. Integrated Algorithm Selection and Configuration: A Case Study for the Modular CMA-ES

Vermetten,

Wang,

Doerr

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

When faced with a specific optimization problem, choosing which algorithm to use is always a tough task. Not only is there a vast variety of algorithms to select from, but these algorithms often are controlled by many hyperparameters, which need to be tuned in order to achieve the best performance possible. Usually, this problem is separated into two parts: algorithm selection and algorithm configuration. With the significant advances made in Machine Learning, however, these problems can be integrated into a combined algorithm selection and hyperparameter optimization task, commonly known as the CASH problem.In this work we compare sequential and integrated algorithm selection and configuration approaches for the case of selecting and tuning the best out of 4,608 variants of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) tested on the Black Box Optimization Benchmark (BBOB) suite. We first show that the ranking of the modular CMA-ES variants depends to a large extent on the quality of the hyperparameters. This implies that even a sequential approach based on complete enumeration of the algorithm space will likely result in sub-optimal solutions. In fact, we show that the integrated approach manages to provide competitive results at a much smaller computational cost.We also compare two different mixed-integer algorithm configuration techniques, called irace and Mixed-Integer Parallel Efficient Global Optimization (MIP-EGO). While we show that the two methods differ significantly in their treatment of the explorationexploitation balance, their overall performances are very similar.

show abstract

“…Online sampling, where the samples are based on solutions encountered by an algorithm during search, has become a popular alternative to random sampling for landscape analysis [60][61][62]. Enhancements to online sampling for continuous spaces include introducing a mechanism for correcting the error produced by the sampling bias [63] and the introduction of path relinking [64].…”

Section: Sampling and Robustness Of Measuresmentioning

confidence: 99%

A Survey of Advances in Landscape Analysis for Optimisation

Malan

2021

Algorithms

View full text Add to dashboard Cite

Fitness landscapes were proposed in 1932 as an abstract notion for understanding biological evolution and were later used to explain evolutionary algorithm behaviour. The last ten years has seen the field of fitness landscape analysis develop from a largely theoretical idea in evolutionary computation to a practical tool applied in optimisation in general and more recently in machine learning. With this widened scope, new types of landscapes have emerged such as multiobjective landscapes, violation landscapes, dynamic and coupled landscapes and error landscapes. This survey is a follow-up from a 2013 survey on fitness landscapes and includes an additional 11 landscape analysis techniques. The paper also includes a survey on the applications of landscape analysis for understanding complex problems and explaining algorithm behaviour, as well as algorithm performance prediction and automated algorithm configuration and selection. The extensive use of landscape analysis in a broad range of areas highlights the wide applicability of the techniques and the paper discusses some opportunities for further research in this growing field.

show abstract

Adaptive landscape analysis

Cited by 24 publications

References 15 publications

Integrated vs. sequential approaches for selecting and tuning CMA-ES variants

Integrated vs. sequential approaches for selecting and tuning CMA-ES variants

Sequential vs. Integrated Algorithm Selection and Configuration: A Case Study for the Modular CMA-ES

A Survey of Advances in Landscape Analysis for Optimisation

Contact Info

Product

Resources

About