A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization

Abstract

In recent years, there has been a growing interest for the experimental analysis in the field of evolutionary algorithms. It is noticeable due to the existence of numerous papers which analyze and propose different types of problems, such as the basis for experimental comparisons of algorithms, proposals of different methodologies in comparison or proposals of use of different statistical techniques in algorithms' comparison.In this paper, we focus our study on the use of statistical techniques in the analysis of evolutionary algorithms' behaviour over optimization problems. A study about the required conditions for statistical analysis of the results is presented by using some models of evolutionary algorithms for real-coding optimization. This study is conducted in two ways: single-problem analysis and multiple-problem analysis. The results obtained state that a parametric statistical analysis could not be appropriate specially when we deal with multiple-problem results. In multiple-problem analysis, we propose the use of non-parametric statistical tests given that they are less restrictive than parametric ones and they can be used over small size samples of results. As a case study, we analyze the published results for the algorithms presented in the

A novel generalized oppositional biogeography-based optimization algorithm: application to peak to average power ratio reduction in OFDM systems

“…Moreover, non parametric statistical tests were conducted in order to further elaborate the algorithms results. These are the Friedman test and the Wilcoxon signed-rank test, which are commonly used for the performance evaluation of EAs [60,61]. Both tests are conducted based on the obtained PAPR values, which are summarized in Tables 4 and 5.…”

Classification Rule Mining with Iterated Greedy

“…Table 2 shows the characteristics of the used 22 datasets from the UCI [20]. Non-parametric tests have been used to compare the results of different algorithms or instances [21]. Specifically, we have considered two alternative methods to analyse the experimental results:…”

Reducing the Complexity of Genetic Fuzzy Classifiers in Highly-Dimensional Classification Problems

“…FeSLiC on the other hand, while it uses this stage conservatively for low to medium feature spaces, it relies on the simplification stage to an extortionate degree for high-dimensional datasets, in order to correct the inability of the rule extraction algorithm to locate the minimum number of informative features. For example, the average number features per [51][52] . The SGERD algorithm has been excluded from this analysis, because it could not be executed in the last three datasets.…”