Open Issues in Surrogate-Assisted Optimization

Stork, Jörg; Friese, Martina; Zaefferer, Martin; Bartz-Beielstein, Thomas; Fischbach, Andreas; Breiderhoff, Beate; Naujoks, Boris; Tušar, Tea

doi:10.1007/978-3-030-18764-4_10

Cited by 25 publications

(18 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A practical and reasonably complete taxonomy of useful techniques is i) metaheuristics such as CMA-ES (Li et al, 2018;Varelas et al, 2018); 4 ii) deterministic algorithms such as DIRECT (Jones & Martins, 2021) or MCS (Huyer & Neumaier, 1999); and iii) surrogate or metamodel-assisted algorithms. In dimension ⪅ 5, metaheuristics and deterministic algorithms are broadly competitive with each other (Sergeyev et al, 2018), but surrogates can yield substantial improvements (Haftka et al, 2016;Vu et al, 2017;Stork et al, 2020;Xia & Shoemaker, 2020).…”

Section: Background On Black-box Optimizationmentioning

confidence: 99%

Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude

Huntsman¹

2022

Preprint

View full text Add to dashboard Cite

Building on the recently developed theory of magnitude, we introduce the optimization algorithm EXPLO2 and carefully benchmark it. EXPLO2 advances the state of the art for optimizing highdimensional (D ⪆ 40) multimodal functions that are expensive to compute and for which derivatives are not available, such as arise in hyperparameter optimization or via simulations. Example 1. Let {x j } 3 j=1 ⊂ R 2 have pairwise distances d jk ∶= d(x j , x k ) given by d 12 = d 13 = 1 = d 21 = d 31 and d 23 = δ = d 32 with δ < 2. It turns out that w 1 = e (δ+2)t − 2e (δ+1)t + e 2t e (δ+2)t − 2e δt + e 2t ;

show abstract

Section: Background On Black-box Optimizationmentioning

confidence: 99%

Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude

Huntsman¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Thus, the surrogate's suitability and accuracy are critical for the optimization's success. Inaccurate surrogate predictions and error estimations, inevitably occurring in large-scale optimization problems, are known to be problematic [23].…”

Section: Surrogate-aidedmentioning

confidence: 99%

“…The existence of numerous variants of surrogate-assisted algorithms indicates that many different ways of using surrogates during optimization [16]). method exist, but also that no best practice procedure has been established yet [23].…”

Section: Introductionmentioning

confidence: 99%

GPSAF: A Generalized Probabilistic Surrogate-Assisted Framework for Constrained Single- and Multi-objective Optimization

Blank¹,

Deb²

2022

Preprint

View full text Add to dashboard Cite

Significant effort has been made to solve computationally expensive optimization problems in the past two decades, and various optimization methods incorporating surrogates into optimization have been proposed. Most research focuses on either exploiting the surrogate by defining a utility optimization problem or customizing an existing optimization method to use one or multiple approximation models. However, only a little attention has been paid to generic concepts applicable to different types of algorithms and optimization problems simultaneously. Thus this paper proposes a generalized probabilistic surrogate-assisted framework (GPSAF), applicable to a broad category of unconstrained and constrained, single-and multi-objective optimization algorithms. The idea is based on a surrogate assisting an existing optimization method. The assistance is based on two distinct phases, one facilitating exploration and another exploiting the surrogates. The exploration and exploitation of surrogates are automatically balanced by performing a probabilistic knockout tournament among different clusters of solutions. A study of multiple well-known population-based optimization algorithms is conducted with and without the proposed surrogate assistance on single-and multi-objective optimization problems with a maximum solution evaluation budget of 300 or less. The results indicate the effectiveness of applying GPSAF to an optimization algorithm and the competitiveness with other surrogate-assisted algorithms.

show abstract

“…In Bayesian optimization (BO), this surrogate supplants the black-box function in the search for which design point(s) to evaluate. This dimensionality-scaling difficulty is mentioned in most existing reviews, see for instance Shan and Wang (2010), Viana et al (2014), Shahriari et al (2016b), Frazier (2018), Ginsbourger (2018) or Stork et al (2020). This article is thus incremental, focusing on more recent trends over several communities (e.g., engineering, operations research, and machine learning) but is by no means exhaustive.…”

Section: Introductionmentioning

confidence: 99%

“…These issues are by-passed byNayebi et al (2019);Moriconi et al (2020), at the cost of the stricter diagonal or axis-aligned embedding assumptions. Changing the domain, as is implicitely done by choosing δ relates to another strategy not explored much in BO: to reduce the optimization space, as discussed byStork et al (2020) or employed with TRs Chen et al (2020a). use a semi-supervised version of sliced inverse regression (SIR) to find important input directions, using both labeled (evaluated) and unlabeled (unevaluated) designs.…”

mentioning

confidence: 99%

A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization

Binois¹,

Wycoff²

2021

Preprint

View full text Add to dashboard Cite

Extending the efficiency of Bayesian optimization (BO) to larger number of parameters has received a lot of attention over the years. Even more so has Gaussian process regression modeling in such contexts, on which most BO methods are based. A variety of structural assumptions have been tested to tame high dimension, ranging from variable selection and additive decomposition to low dimensional embeddings and beyond. Most of these approaches in turn require modifications of the acquisition function optimization strategy as well. Here we review the defining assumptions, and discuss the benefits and drawbacks of these approaches in practice.

show abstract

Open Issues in Surrogate-Assisted Optimization

Cited by 25 publications

References 59 publications

Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude

Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude

GPSAF: A Generalized Probabilistic Surrogate-Assisted Framework for Constrained Single- and Multi-objective Optimization

A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization

Contact Info

Product

Resources

About