The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms

Abstract

The Stochastic Optimisation Software (SOS) is a Java platform facilitating the algorithmic design process and the evaluation of metaheuristic optimisation algorithms. It reduces the burden of coding miscellaneous methods for dealing with several bothersome and time-demanding tasks such as parameter tuning, implementation of comparison algorithms and testbed problems, collecting and processing data to display results, measuring algorithmic overhead, etc. SOS provides numerous off-the-shelf methods including 1) customised implementations of statistical tests, such as the Wilcoxon Rank-Sum test and the Holm-Bonferroni procedure, for comparing performances of optimisation algorithms and automatically generate result tables in PDF and LaTeX formats; 2) the implementation of an original advanced statistical routine for accurately comparing couples of stochastic optimisation algorithms; 3) the implementation of a novel testbed suite for continuous optimisation, derived from the IEEE CEC 2014 benchmark, allowing for controlled activation of the rotation operator. each testbed function. Moreover, this article comments on the current state of the literature in stochastic optimisation and highlights similarities shared by modern metaheuristics inspired by nature. It is argued that the vast majority of these algorithms are simply a reformulation of the same methods and that metaheuristics for optimisation should be simply treated as stochastic processes with less emphasis on the inspiring metaphor behind them.

Can Single Solution Optimisation Methods Be Structurally Biased?

“…A brief description of the employed methods is reported in the reminder of this section. These optimisation methods are implemented and tested, with the parameter configurations suggested in their original papers, using the software platform [11]. Details on their implementations can be obtained from the source code made available in the online repository [24], thus facilitating the replicability of the results presented in this study.…”

“…As more than abundant literature [11], [12] suggests, optimisation methods can be compared in the light of different aspects: best/average/worst performance, complexity, universality of application, memory usage, scalability, etc. Moreover, performance can be evaluated on a class of functions (with widely varying definitions of 'class') or over all possible problems (which is practically impossible but theoretically relevant in the context of global convergence proofs).…”

“…This superficial correction/repair strategy is inherent to the method and rarely encountered in the field. All simulations are carried out in the SOS platform [11]. The source of all algorithms and the necessary experimental setup for reproducing the presented results are available in [24].…”

Can Compact Optimisation Algorithms Be Structurally Biased?

“…All algorithms refer to their persistent elitist variants. All experiments are executed on a standard desktop using the SOS platform [3] implemented in Java (algorithms' source code is available online). It is worth mentioning that the aforementioned pseudorandom generator used for all experiments is considered on the better side of the scale for linear congruential generators [15] 4 Discussion of results…”

“…The field of EAs is saturated with a multitude of nature inspired algorithms [2,3]. For practical reasons, these algorithms need to be compared and characterised.…”