Benchmarking large-scale continuous optimizers: The bbob-largescale testbed, a COCO software guide and beyond

Abstract

Benchmarking of optimization solvers is an important and compulsory task for performance assessment that in turn can help in improving the design of algorithms. It is a repetitive and tedious task. Yet, this task has been greatly automatized in the past ten years with the development of the Comparing Continuous Optimizers platform (COCO). In this context, this paper presents a new testbed, called bbob-largescale, that contains functions ranging from dimension 20 to 640, compatible with and extending the well-known single-objective noiseless bbob test suite to larger dimensions. The test suite contains 24 single-objective functions in continuous domain, built to model well-known difficulties in continuous optimization and to test the scaling behavior of algorithms. To reduce the computational demand of the orthogonal search space transformations that appear in the bbob test suite, while retaining some desired properties, we use permuted block diagonal orthogonal matrices. The paper discusses implementation technicalities and presents a guide for using the test suite within the COCO platform and for interpreting the postprocessed output. The source code of the new test suite is available on GitHub as part of the open source COCO benchmarking platform.

Towards exploratory landscape analysis for large-scale optimization

“…We believe that the time to call a flacco function from pflacco is negligible when 𝑛 is large enough. We used the BBOB function set [11] for 𝑛 ∈ {2, 3, 5, 10} and its large-scale version [48] for 𝑛 ∈ {20, 40, 80, 160, 320, 640}. Both function sets are available in the COCO platform [10].…”

“…In this context, first, we investigate the computation time of features in the R-package flacco [20], which currently provides 17 feature classes. We use the BBOB function set and its large-scale version [48], which consists of the 24 functions with 𝑛 ∈ {20, 40, 80, 160, 320, 640}. The computation time of the ELA features has been paid little attention in the literature.…”

“…Each BBOB function further consists many instances. The large-scale version of the BBOB function set [48] provides problems with up to 𝑛 = 640.…”

IOHanalyzer: Detailed Performance Analyses for Iterative Optimization Heuristics

“…In evolutionary computation, the arguably best established benchmarking environment is the already mentioned COCO platform [30]. Originally designed to compare derivative-free optimization algorithms operating on numeric optimization problems [29], the tool has seen several extensions in the last years, e.g., towards multiobjective optimization [58], mixed-integer optimization [57], and large-scale optimization [60]. COCO consists of an experimentation part that produces data iles with detailed performance traces, and an automated data analysis part in which a ixed number of standardized analyses are automatically generated.…”

Kısıtlı Mühendislik Problemlerinin Karşılaştırmalı Ağırlık ve Maliyet Optimizasyonu

“…Algoritma performansını ölçmenin en önemli yolu algoritmaları benchmark problemleri üzerinden kıyaslamaktır. [28][29][30][31] çalışmalarında çeşitli benchmark problemleri üzerinden optimizasyon algoritmalarını kıyas-lamışlardır. Birçok gerçek mühendislik problemlerindeki ana karmaşıklık ise problemin boyutu, kısıtlayıcılara sahip olması ve değişkenlerin birbirleri ile etkileşiminden kaynaklanır [32][33][34][35][36].…”