Constant optimization and feature standardization in multiobjective genetic programming

Abstract

This paper extends the numerical tuning of tree constants in genetic programming (GP) to the multiobjective domain. Using ten real-world benchmark regression datasets and employing Bayesian comparison procedures, we first consider the effects of feature standardization (without constant tuning) and conclude that standardization generally produces lower test errors, but, contrary to other recently published work, we find much less clear trend for tree sizes. In addition, we consider the effects of constant tuning – with and without feature standardization – and observe that (1) constant tuning invariably improves test error, and (2) usually decreases tree size. Combined with standardization, constant tuning produces the best test error results; tree sizes, however, are increased. We also examine the effects of applying constant tuning only once at the end a conventional GP run which turns out to be surprisingly promising. Finally, we consider the merits of using numerical procedures to tune tree constants and observe that for around half the datasets evolutionary search alone is superior whereas for the remaining half, parameter tuning is superior. We identify a number of open research questions that arise from this work.

Comprehensive analysis of Classical Machine Learning models and Ensemble methods for predicting Crime in urban society

“…• Standardization: Feature standardization drives the values of individual features in the data to have zero mean and zero-unit variance (Liu, 2020). The known computation technique is to decide the mean distribution and each feature's standard deviation and estimate the latest data point by the given formula (Rockett, 2022):…”