Symbolic Regression by Exhaustive Search: Reducing the Search Space Using Syntactical Constraints and Efficient Semantic Structure Deduplication

Abstract

Symbolic regression is a powerful system identification technique in industrial scenarios where no prior knowledge on model structure is available. Such scenarios often require specific model properties such as interpretability, robustness, trustworthiness and plausibility, that are not easily achievable using standard approaches like genetic programming for symbolic regression. In this chapter we introduce a deterministic symbolic regression algorithm specifically designed to address these issues. The algorithm uses a context-free grammar to produce models that are parameterized by a non-linear least squares local optimization procedure. A finite enumeration of all possible models is guaranteed by structural restrictions as well as a caching mechanism for detecting semantically equivalent solutions. Enumeration order is established via heuristics designed to improve search efficiency. Empirical tests on a comprehensive benchmark suite show that our approach is competitive with genetic programming in many noiseless problems while maintaining desirable properties such as simple, reliable models and reproducibility.

A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge

“…Furthermore, these systems tend to overfit given large and noisy data 41 , which is the case of typical empirical results in physics. Two main methods to overcome the computational expense are performed by 42,43 , where they apply a brute-force approach on a reduced search space rather than perform an incomplete search in the entire search space. In both methods, the search space is reduced by removing algebraically equivalent expressions, either through the recursive application of the grammar production rules 42 or by preventing semantic duplicates using grammar restrictions and semantic hashing 43 .…”

A new imputation method based on genetic programming and weighted KNN for symbolic regression with incomplete data

“…However, several control parameters should be tuned in the EBR method. In [16], a deterministic symbolic regression algorithm is proposed using a context-free grammar. It utilises non-linear least squares local optimisation to produce the symbolic regression models.…”

Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristics

“…Individuals with high scores have a higher probability of being selected for the next iteration of crossover, mutation, and reproduction. Ideas to enhance genetic algorithms have been proposed in [11,12] to reduce the search space. Nicolau and McDermott [14] use prior information of the values of the dependent variable.…”

“…The quality of solutions is not stable, however, because genetic algorithms are a stochastic process, which means that it can generate different solutions for the same input data and the same settings. Kammerer et al [11] remark that "it might produce highly dissimilar solutions even for the same input data. "…”