Lightweight Symbolic Regression with the Interaction - Transformation Representation

Aldeia, Guilherme Seidyo Imai; França, Fabrício Olivetti de

doi:10.1109/cec.2018.8477951

Cited by 9 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative representation was proposed in [9,10] constraining the search space to a specific pattern composed of an affine combination of nonlinear transformed features. These features are a composition of a polynomial, representing the interactions between the original variables, and the application of an unary function, called transformation.…”

Section: Introductionmentioning

confidence: 99%

Multi-Objective Optimization with Transformation-Interaction-Rational Representation for Symbolic Regression

França

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The Transformation-Interaction-Rational is a representation for symbolic regression that limits the search space of function forms to well structured functions enabling the use of efficient numerical coefficient determination. The performance of using Genetic Programming with this representation was substantially better than with its predecessor (Interaction-Transformation) while ranking close to the state-of-the-art on a contemporary Symbolic Regression benchmark. In this same benchmark, we observed that the performance could be further improved with an additional selective pressure on smaller datasets for the expression size, thus creating smaller expressions and generalizing better. This was verified with a penalization term applied to the fitness measure together with heuristic rules to determine what is a small dataset. Possibly a better strategy is to adopt a multi-objective search to find a set of non-dominated solutions taking accuracy and model size in consideration. This will automatically stimulate the preference for simpler models whenever two equally accurate model exists, possibly the need for heuristic rules and penalty functions. In this paper, we extend the evolution library used by TIR to support multi-objective optimization, specifically the NSGA-II algorithm, and apply that to the same benchmark. The results show that the use of multi-objective optimization improves the results on a selection of the benchmarks increasing the overall rank of the algorithm to a second place in the benchmark. However, this strategy is still sensitive to overfitting on small data sets.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Objective Optimization with Transformation-Interaction-Rational Representation for Symbolic Regression

França

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another possible solution is to create a constrained representation powerful enough to fit nonlinear relations but restrictive to the point of making it impossible to have some complex constructs. A recent example of this approach, which is extended in this article, is the Interaction-Transformation representation (Aldeia and de Franca, 2018;de Franca, 2018). This representation together with the search heuristic introduced in that article will be explained in the next section.…”

Section: Introductionmentioning

confidence: 99%

Interaction–Transformation Evolutionary Algorithm for Symbolic Regression

França

Aldeia

2021

Evolutionary Computation

View full text Add to dashboard Cite

Interaction-Transformation (IT) is a new representation for Symbolic Regression that reduces the space of solutions to a set of expressions that follow a specific structure. The potential of this representation was illustrated in prior work with the algorithm called SymTree. This algorithm starts with a simple linear model and incrementally introduces new transformed features until a stop criterion is met. While the results obtained by this algorithm were competitive with the literature, it had the drawback of not scaling well with the problem dimension. This paper introduces a mutation only Evolutionary Algorithm, called ITEA, capable of evolving a population of IT expressions. One advantage of this algorithm is that it enables the user to specify the maximum number of terms in an expression. In order to verify the competitiveness of this approach, ITEA is compared to linear, nonlinear and Symbolic Regression models from the literature. The results indicate that ITEA is capable of finding equal or better approximations than other Symbolic Regression models while being competitive to state-of-the-art non-linear models. Additionally, since this representation follows a specific structure, it is possible to extract the importance of each original feature of a data set as an analytical function, enabling us to automate the explanation of any prediction. In conclusion, ITEA is competitive when comparing to regression models with the additional benefit of automating the extraction of additional information of the generated models.

show abstract

“…Another possible solution is to create a constrained representation powerful enough to fit nonlinear relations but restrictive to the point of making it impossible to have some complex constructs. A recent example of this approach, which is extended in this paper, is the Interaction-Transformation representation [2,7]. This representation together with the seminal search heuristic will be explained in the next section.…”

Section: Introductionmentioning

confidence: 99%

Interaction-Transformation Evolutionary Algorithm for Symbolic Regression

França¹,

Aldeia²

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Interaction-Transformation (IT) is a new representation for Symbolic Regression that restricts the search space into simpler expressions. While this representation is limited to a subspace of the commonly used expression trees, previous work showed that, when coupled with a simple search-based algorithm, the found expressions were expressive enough to compete with commonly used regression algorithms while maintaining the simplicity. The main drawback of the proposed search algorithm was that the size of the generated expressions grew excessively large in some cases. This paper introduces an Evolutionary Algorithm capable of evolving a population of IT expressions supported only by a mutation operator with the main advantage of controlling some aspects of the generated expressions like their size, for example. Some experimentation is performed consisting of 8 commonly used data sets all divided into training and test sets through a 5-fold procedure. The proposed algorithm is then compared to 4 linear models and 5 symbolic regression algorithms in terms of approximation and generalization error. Additionally, the interpretability of the proposed approach is illustrated and compared to one of the competitors. The results show evidence that this representation is capable of finding better approximations to real-world data sets when compared to linear regression models and state-of-the-art Genetic Programming algorithms. Also, since the IT expressions follow a specified format, it is possible to extract the behavior information of each input variable w.r.t. the generated model. In conclusion, this new algorithm is competitive to the state-of-the-art and capable of generating interpretable expressions.

show abstract

Lightweight Symbolic Regression with the Interaction - Transformation Representation

Cited by 9 publications

References 14 publications

Multi-Objective Optimization with Transformation-Interaction-Rational Representation for Symbolic Regression

Multi-Objective Optimization with Transformation-Interaction-Rational Representation for Symbolic Regression

Interaction–Transformation Evolutionary Algorithm for Symbolic Regression

Interaction-Transformation Evolutionary Algorithm for Symbolic Regression

Contact Info

Product

Resources

About