Latent Variable Symbolic Regression for High-Dimensional Inputs

McConaghy, Trent

doi:10.1007/978-1-4419-1626-6_7

Cited by 7 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [12], the authors proposed a technique based on latent variables, non-linear sensitivity analysis, and GP to manage approximation problems when the number of input variables is high. The proposed technique was tested with 340 input variable problems.…”

Section: Related Workmentioning

confidence: 99%

Transformation of input space using statistical moments: EA-based approach

Kattan¹,

Kampouridis

Ong

et al. 2014

2014 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

Abstract-Standard Regression models are presented with n samples from an input space χ that is composed of observational data of the form (xi, y(xi)), i = 1...n where each xi denotes a k-dimensional input vector of design variables and y is the response. When k n, high variance and over-fitting become a major concern. In this paper we propose a novel approach to mitigate this problem by transforming the input vectors into new smaller vectors (called Z set) using only a set of simple statistical moments. Genetic Algorithm (GA) has been used to evolve a transformation procedure. It is used to optimise an optimal sequence of statistical moments and their input parameters. We used Linear Regression (LR) as an example to quantify the quality of the evolved transformation procedure. Empirical evidences, collected from benchmark functions and real-world problems, demonstrate that the proposed transformation approach is able to dramatically improve LR generalisation and make it outperform other state-of-the-art regression models such as Genetic Programming, Kriging, and Radial Basis Functions Networks. In addition, we present an analysis to shed light on the most important statistical moments that are useful for the transformation process.

show abstract

Section: Related Workmentioning

confidence: 99%

Transformation of input space using statistical moments: EA-based approach

Kattan¹,

Kampouridis

Ong

et al. 2014

2014 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

show abstract

“…In [8] the authors proposed a technique based on latent variables, non-linear sensitivity analysis, and GP to manage approximation problems when the number of input variables is high. The proposed technique was tested with 340 input variable problems.…”

Section: Related Workmentioning

confidence: 99%

“…One way to mitigate the curse of dimensionality problem is by reducing the number of design variables using some dimensionality reduction technique such as Principle Component Analysis (PCA) or Factor Analysis (FA) (e.g., see [8]). However, variables reduction is reasonable only when the significant variables are just a fraction of the overall set of variables.…”

Section: Introductionmentioning

confidence: 99%

Generalisation Enhancement via Input Space Transformation: A GP Approach

Kattan¹,

Kampouridis²,

Agapitos³

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper proposes a new approach to improve generalisation of standard regression techniques when there are hundreds or thousands of input variables. The input space X is composed of observational data of the form (xi, y(xi)), i = 1...n where each xi denotes a k-dimensional input vector of design variables and y is the response. Genetic Programming (GP) is used to transform the original input space X into a new input space Z = (zi, y(zi)) that has smaller input vector and is easier to be mapped into its corresponding responses. GP is designed to evolve a function that receives the original input vector from each xi in the original input space as input and return a new vector zi as an output. Each element in the newly evolved zi vector is generated from an evolved mathematical formula that extracts statistical features from the original input space. To achieve this, we designed GP trees to produce multiple outputs. Empirical evaluation of 20 different problems revealed that the new approach is able to significantly reduce the dimensionality of the original input space and improve the performance of standard approximation models such as Kriging, Radial Basis Functions Networks, and Linear Regression, and GP (as a regression techniques). In addition, results demonstrate that the new approach is better than standard dimensionality reduction techniques such as Principle Component Analysis (PCA). Moreover, the results show that the proposed approach is able to improve the performance of standard Linear Regression and make it competitive to other stochastic regression techniques.

show abstract

“…In [154], symbolic regression on data sets with hundreds of features is addressed using non-traditional GP technique. This technique is designed based on GP, latent variables, and nonlinear sensitivity analysis.…”

Section: High-dimensional Symbolic Regressionmentioning

confidence: 99%

Genetic Programming for Symbolic Regression on Incomplete Data

Al-Helali¹

View full text Add to dashboard Cite

Symbolic regression is the process of constructing mathematical expressions that best fit given data sets, where a target variable is expressed in terms of input variables. Unlike traditional regression methods, which optimise the parameters of pre-defined models, symbolic regression learns both the model structure and its parameters simultaneously. Genetic programming (GP) is a biologically-inspired evolutionary algorithm, that automatically generates computer programs to solve a given task. The flexible representation of GP along with its ``white box" nature makes it a dominant method for symbolic regression. Moreover, GP has been successfully employed for different learning tasks such as feature selection and transfer learning. Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general, especially when dealing with real-world data sets. One common approach to handling data missingness is data imputation. Data imputation is the process of estimating missing values based on existing data. Another approach to deal with incomplete data is to build learning algorithms that directly work with missing values. Although a number of methods have been proposed to tackle the data missingness issue in machine learning, most studies focus on classification tasks. Little attention has been paid to symbolic regression on incomplete data. The existing symbolic regression methods are only applicable when the given data set is complete. The overall goal of the thesis is to improve the performance of symbolic regression on incomplete data by using GP for data imputation, instance selection, feature selection, and transfer learning. This thesis develops an imputation method to handle missing values for symbolic regression. The method integrates the instance-based similarity of the k-nearest neighbour method with the feature-based predictability of GP to estimate the missing values. The results show that the proposed method outperforms existing popular imputation methods. This thesis develops an instance selection method for improving imputation for symbolic regression on incomplete data. The proposed method has the ability to simultaneously build imputation and symbolic regression models such that the performance is improved. The results show that involving instance selection with imputation advances the performance of using the imputation alone. High-dimensionality is a serious data challenge, which is even more difficult on incomplete data. To address this problem in symbolic regression tasks, this thesis develops a feature selection method that can select a good set of features directly from incomplete data. The method not only improves the regression accuracy, but also enhances the efficiency of symbolic regression on high-dimensional incomplete data. Another challenging problem is data shortage. This issue is even more challenging when the data is incomplete. To handle this situation, this thesis develops transfer learning methods to improve symbolic regression in domains with incomplete and limited data. These methods utilise two powerful abilities of GP: feature construction and feature selection. The results show the ability of these methods to achieve positive transfer learning from domains with complete data to different (but related) domains with incomplete data. In summary, the thesis develops a range of approaches to improving the effectiveness and efficiency of symbolic regression on incomplete data by developing a number of GP-based methods. The methods are evaluated using different types of data sets considering various missingness and learning scenarios.

show abstract

Latent Variable Symbolic Regression for High-Dimensional Inputs

Cited by 7 publications

References 21 publications

Transformation of input space using statistical moments: EA-based approach

Transformation of input space using statistical moments: EA-based approach

Generalisation Enhancement via Input Space Transformation: A GP Approach

Genetic Programming for Symbolic Regression on Incomplete Data

Contact Info

Product

Resources

About