Analysing symbolic regression benchmarks under a meta-learning approach

Abstract

e de nition of a concise and e ective testbed for Genetic Programming (GP) is a recurrent ma er in the research community.is paper takes a new step in this direction, proposing a di erent approach to measure the quality of the symbolic regression benchmarks quantitatively. e proposed approach is based on meta-learning and uses a set of dataset meta-features-such as the number of examples or output skewness-to describe the datasets. Our idea is to correlate these meta-features with the errors obtained by a GP method. ese meta-features de ne a space of benchmarks that should, ideally, have datasets (points) covering di erent regions of the space. An initial analysis of 63 datasets showed that current benchmarks are concentrated in a small region of this benchmark space. We also found out that number of instances and output skewness are the most relevant meta-features to GP output error. Both conclusions can help de ne which datasets should compose an e ective testbed for symbolic regression methods.

Genetic programming benchmarks

“…Nicolau et al (2015) offered advice on choice of problems, noise, train-tests splits, and more, in the context of symbolic regression. Oliveira et al (2018) defined a meta-space of benchmark properties with the goal of finding areas in the space not covered by any benchmarks. Woodward et al (2014) discussed the issue of mismatch between benchmarks and real-world problems.…”