Parallelized Bayesian Optimization for Expensive Robot Controller Evolution

Rebolledo, Margarita; Rehbach, Frederik; Eiben, A. E.; Bartz-Beielstein, Thomas

doi:10.1007/978-3-030-58112-1_17

“…While it is only rarely able to compete in a scenario with a batch size of two, it can compete on a total of ten function/dimension combinations with a batch size of 16. This coincides with other experiments that have shown that a CMAES starts outperforming modelbased methods when it is given enough budget [30]. This observation is further signified considering Figure 6.…”

Section: A Static Batch Size Experimentssupporting

confidence: 90%

Benchmark-Driven Algorithm Configuration Applied to Parallel Model-Based Optimization

Rehbach¹,

Zaefferer²,

Fischbach³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

<div>This paper introduces a benchmarking framework that allows rigorous evaluation of parallel model-based optimizers for expensive functions.</div><div>The framework establishes a relationship between estimated costs of parallel function evaluations (on real-world problems) to known sets of test functions.</div><div>Such real-world problems are not always readily available (e.g., confidentiality, proprietary software).</div><div>Therefore, new test problems are created by Gaussian process simulation. </div><div>The proposed framework is applied in an extensive benchmark study to compare multiple state-of-the-art parallel optimizers with a novel model-based algorithm, which combines ideas of an explorative search for global model quality with parallel local searches to increase function exploitation. </div><div>The benchmarking framework is used to configure good batch size setups for parallel algorithms systematically based on landscape properties.</div><div>Furthermore, we introduce a proof-of-concept for a novel automatic batch size configuration.</div><div>The predictive quality of the batch size configuration is evaluated on a large set of test functions and the functions generated by Gaussian process simulation. </div><div>The introduced algorithm outperforms multiple state-of-the-art optimizers, especially on multi-modal problems.</div><div>Additionally, it proves to be particularly robust over various problem landscapes, and performs well with all tested batch sizes. </div><div>Consequently, this makes it well-suited for black-box kinds of problems. </div>

show abstract

“…Rebolledo et al [30] showed that parallelization can also be used in situations where the best choice of kernel, model, or infill criteria is unknown. A number of samples can be generated by building multiple differently configured models and optimizing them with different infill criteria in parallel.…”

Section: A Related Literaturementioning

confidence: 99%

Benchmark-Driven Algorithm Configuration Applied to Parallel Model-Based Optimization

Rehbach¹,

Zaefferer²,

Fischbach³

et al. 2022

Preprint

Self Cite

1

0

View full text Add to dashboard Cite

<div>This paper introduces a benchmarking framework that allows rigorous evaluation of parallel model-based optimizers for expensive functions.</div><div>The framework establishes a relationship between estimated costs of parallel function evaluations (on real-world problems) to known sets of test functions.</div><div>Such real-world problems are not always readily available (e.g., confidentiality, proprietary software).</div><div>Therefore, new test problems are created by Gaussian process simulation. </div><div>The proposed framework is applied in an extensive benchmark study to compare multiple state-of-the-art parallel optimizers with a novel model-based algorithm, which combines ideas of an explorative search for global model quality with parallel local searches to increase function exploitation. </div><div>The benchmarking framework is used to configure good batch size setups for parallel algorithms systematically based on landscape properties.</div><div>Furthermore, we introduce a proof-of-concept for a novel automatic batch size configuration.</div><div>The predictive quality of the batch size configuration is evaluated on a large set of test functions and the functions generated by Gaussian process simulation. </div><div>The introduced algorithm outperforms multiple state-of-the-art optimizers, especially on multi-modal problems.</div><div>Additionally, it proves to be particularly robust over various problem landscapes, and performs well with all tested batch sizes. </div><div>Consequently, this makes it well-suited for black-box kinds of problems. </div>

show abstract