An Empirical Study of Functional Complexity as an Indicator of Overfitting in Genetic Programming

Trujillo, Leonardo; Silva, Sara; Legrand, Pierrick; Vanneschi, Leonardo

doi:10.1007/978-3-642-20407-4_23

Cited by 13 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A complexity measure based on slope of line segments is proposed in [29]. The slope-based functional complexity (SFC) is computed by taking sum of differences of slope of consecutive line segments.…”

Section: ) Behavioral Complexitymentioning

confidence: 99%

“…The slope-based functional complexity (SFC) is computed by taking sum of differences of slope of consecutive line segments. However, authors [29] calculated SFC measure for each problem dimensions separately in case of multi-dimensional problems. To overcome limitations of SFC, a new measure based on concept of measuring amount of variation in output is presented in [29].…”

Section: ) Behavioral Complexitymentioning

confidence: 99%

See 1 more Smart Citation

Empirical modeling using genetic programming: a survey of issues and approaches

Dabhi

Chaudhary

2014

Nat Comput

View full text Add to dashboard Cite

Abstract--In the field of empirical modeling using Genetic Programming (GP), it is important to evolve solution with good generalization ability. Generalization ability of GP solutions get affected by two important issues: bloat and over-fitting. We surveyed and classified existing literature related to different techniques used by GP research community to deal with these issues. We also point out limitation of these techniques, if any. Moreover, the classification of different bloat control approaches and measures for bloat and over-fitting are also discussed. We believe that this work will be useful to GP practitioners in following ways: (i) to better understand concepts of generalization in GP (ii) comparing existing bloat and over-fitting control techniques and (iii) selecting appropriate approach to improve generalization ability of GP evolved solutions.

show abstract

Section: ) Behavioral Complexitymentioning

confidence: 99%

Section: ) Behavioral Complexitymentioning

confidence: 99%

Empirical modeling using genetic programming: a survey of issues and approaches

Dabhi

Chaudhary

2014

Nat Comput

View full text Add to dashboard Cite

show abstract

“…However, recent research has observed that overfitting can occur in absence of bloat [41,67]. Therefore, some researchers suggested that bloat and overfitting are two independent phenomena and should be tackled by separate mechanisms [67,217].…”

Section: List Of Publicationsmentioning

confidence: 99%

“…However, measuring the complexity of the candidate models is not a trivial task. A number of model complexity measures for GP have been proposed [42,217,224,233]. Some of them use the complexity of solutions as an indicator of overfitting, while others treat the complexity as an objective that needs to be optimised.…”

Section: List Of Publicationsmentioning

confidence: 99%

“…The results indicated that small, generalisable solutions might be difficult to find when a simple size penalty was applied. More recently, researchers found that, instead of shorter solutions, behaviourally/functionally simpler solutions are more likely to generalise better [78,217,224,233]. The idea of quantitatively studying the relationship between generalisation ability and model complexity have been approached by several works.…”

Section: Restricting Model Complexity In Gpmentioning

confidence: 99%

See 1 more Smart Citation

Improving the Generalisation of Genetic Programming for Symbolic Regression

Chen¹

View full text Add to dashboard Cite

<p>Symbolic regression (SR) is a function identification process, the task of which is to identify and express the relationship between the input and output variables in mathematical models. SR is named to emphasise its ability to find the structure and coefficients of the model simultaneously. Genetic Programming (GP) is an attractive and powerful technique for SR, since it does not require any predefined model and has a flexible representation. However, GP based SR generally has a poor generalisation ability which degrades its reliability and hampers its applications to science and real-world modeling. Therefore, this thesis aims to develop new GP approaches to SR that evolve/learn models exhibiting good generalisation ability. This thesis develops a novel feature selection method in GP for high-dimensional SR. Feature selection can potentially contribute not only to improving the efficiency of learning algorithms but also to enhancing the generalisation ability. However, feature selection is seldom considered in GP for high-dimensional SR. The proposed new feature selection method utilises GP’s built-in feature selection ability and relies on permutation to detect the truly relevant features and discard irrelevant/noisy features. The results confirm the superiority of the proposed method over the other examined feature selection methods including random forests and decision trees on identifying the truly relevant features. Further analysis indicates that the models evolved by GP with the proposed feature selection method are more likely to contain only the truly relevant features and have better interpretability. To address the overfitting issue of GP when learning from a relatively small number of instances, this thesis proposes a new GP approach by incorporating structural risk minimisation (SRM), which is a framework to estimate the generalisation performance of models, into GP. The effectiveness of SRM highly depends on the accuracy of the Vapnik-Chervonenkis (VC) dimension measuring model complexity. This thesis significantly extends an experimental method (instead of theoretical estimation) to measure the VC-dimension of a mixture of linear and nonlinear regression models in GP for the first time. The experimental method has been conducted using uniform and non-uniform settings and provides reliable VC-dimension values. The results show that our methods have an impressively better generalisation gain and evolve more compact model, which have a much smaller behavioural difference from the target models than standard GP and GP with bootstrap, The proposed method using the optimised non-uniform setting further improves the one using the uniform setting. This thesis employs geometric semantic GP (GSGP) to tackle the unsatisfied generalisation performance of GP for SR when no overfitting occurs. It proposes three new angle-awareness driven geometric semantic operators (GSO) including selection, crossover and mutation to further explore the geometry of the semantic space to gain a greater generalisation improvement in GP for SR. The angle-awareness brings new geometric properties to these geometric operators, which are expected to provide a greater leverage for approximating the target semantics in each operation, and more importantly, to be resistant to overfitting. The results show that compared with two kinds of state-of-the-art GSOs, the proposed new GSOs not only drive the evolutionary process fitting the target semantics more efficiently but also significantly improve the generalisation performance. A further comparison on the evolved models shows that the new method generally produces simpler models with a much smaller size and containing important building blocks of the target models.</p>

show abstract

A Discrete Cosine Transform Based Evolutionary Algorithm and Its Application for Symbolic Regression

Liu

2019

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

An Empirical Study of Functional Complexity as an Indicator of Overfitting in Genetic Programming

Cited by 13 publications

References 11 publications

Empirical modeling using genetic programming: a survey of issues and approaches

Empirical modeling using genetic programming: a survey of issues and approaches

Improving the Generalisation of Genetic Programming for Symbolic Regression

A Discrete Cosine Transform Based Evolutionary Algorithm and Its Application for Symbolic Regression

Contact Info

Product

Resources

About