Continuous Optimization Benchmarks by Simulation

Zaefferer, Martin; Rehbach, Frederik

doi:10.1007/978-3-030-58112-1_19

Cited by 7 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We proposed the generation of data-driven test functions based on Kriging simulations to circumvent this problem. This extends methods that were previously described by Zaefferer and Rehbach [40]. The methods are now publicly available in the form of the R-package 'COBBS' (Continuous Optimization Benchmarks By Simulation).…”

Section: Discussionmentioning

confidence: 95%

“…Test function generation by simulation tries to alleviate this problem. Previous work describes a method for the generation of continuous optimization benchmark functions via the simulation of Kriging models [39], [40]. Details of Kriging prediction and simulation are, e.g., provided by Cressie [7].…”

Section: Simulation-based Benchmarksmentioning

confidence: 99%

“…The core idea of Continuous Optimization Benchmarks by Simulation (COBBS) is to generate new problem instances from existing data, especially in contexts where 'real' problem instances (i.e., the corresponding objective functions) are unavailable or too expensive for an in-detail experimental investigation such as a benchmarking study [40].…”

Section: Simulation-based Benchmarksmentioning

confidence: 99%

“…In this work, we will add an extension to the Kriging simulation in COBBS. The Kriging model used by Zaefferer and Rehbach [40] is a stationary one, with a constant trend. This will often limit the quality of the model, and hence, the quality of the produced test instances.…”

Section: Simulation-based Benchmarksmentioning

confidence: 99%

“…The samples stem from the previous optimization runs on those functions. We use the same approach of generating training data as described in [40] because the samples a) cover the whole search space in a reasonable manner and b) cover potentially optimal areas more densely, thus being able to represent both local and global patterns in the data.…”

Section: Benchmarking the Adaptive Algorithm 1) Benchmarking On Bbobmentioning

confidence: 99%

See 4 more Smart Citations

Benchmark-Driven Algorithm Configuration Applied to Parallel Model-Based Optimization

Rehbach¹,

Zaefferer²,

Fischbach³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

<div>This paper introduces a benchmarking framework that allows rigorous evaluation of parallel model-based optimizers for expensive functions.</div><div>The framework establishes a relationship between estimated costs of parallel function evaluations (on real-world problems) to known sets of test functions.</div><div>Such real-world problems are not always readily available (e.g., confidentiality, proprietary software).</div><div>Therefore, new test problems are created by Gaussian process simulation. </div><div>The proposed framework is applied in an extensive benchmark study to compare multiple state-of-the-art parallel optimizers with a novel model-based algorithm, which combines ideas of an explorative search for global model quality with parallel local searches to increase function exploitation. </div><div>The benchmarking framework is used to configure good batch size setups for parallel algorithms systematically based on landscape properties.</div><div>Furthermore, we introduce a proof-of-concept for a novel automatic batch size configuration.</div><div>The predictive quality of the batch size configuration is evaluated on a large set of test functions and the functions generated by Gaussian process simulation. </div><div>The introduced algorithm outperforms multiple state-of-the-art optimizers, especially on multi-modal problems.</div><div>Additionally, it proves to be particularly robust over various problem landscapes, and performs well with all tested batch sizes. </div><div>Consequently, this makes it well-suited for black-box kinds of problems. </div>

show abstract

Section: Discussionmentioning

confidence: 95%