Using Shape Constraints for Improving Symbolic Regression Models

Abstract

We describe and analyze algorithms for shape-constrained symbolic regression, which allows the inclusion of prior knowledge about the shape of the regression function. This is relevant in many areas of engineering -in particular whenever a data-driven model obtained from measurements must have certain properties (e.g. positivity, monotonicity or convexity/concavity). We implement shape constraints using a soft-penalty approach which uses multiobjective algorithms to minimize constraint violations and training error. We use the non-dominated sorting genetic algorithm (NSGA-II) as well as the multi-objective evolutionary algorithm based on decomposition (MOEA/D). We use a set of models from physics textbooks to test the algorithms and compare against earlier results with single-objective algorithms. The results show that all algorithms are able to find models which conform to all shape constraints. Using shape constraints helps to improve extrapolation behavior of the models.

Shape-constrained Symbolic Regression with NSGA-III

“…In this work two multi/many-objective evolutionary search frameworks, as described in [5], are used. Both presented algorithm start with a random initialized population, represented as expression trees.…”

“…Both algorithms are configured equally: max number of evaluation 500000, tournament selection with a crowded group size of 5, single-subtree crossover, and a mutator that either changes a single node or a subtree with a randomly initialized subtree. Further parameter settings can be taken from [5].…”

“…For example, physical laws such as conservation of energy, thermodynamics, or laws of gravity can be enforced for the modeling process when only a few data points are given or when they are biased to prescribe some expected behavior. Since a desired shape of the function form is enforced, these constraints are called shape-constraints [4,5,14].…”