A Hybrid GP-KNN Imputation for Symbolic Regression with Missing Values

Al-Helali, Baligh; Chen, Qi; Xue, Bing; Zhang, Mengjie

doi:10.1007/978-3-030-03991-2_33

Cited by 23 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…bias of estimating the prediction inaccuracy(Noei and Abadeh, 2019) Bayesian principal component analysis (BPCA)þ local least squares Better performance(Li et al, 2015) Classification-based imputation Robust missing data imputation model for single-cell RNA-seq data(Mesquita and Gomes, 2017) Better imputation accuracy with ordinal data(Yang et al, 2018) Genetic programming þ k-nearest neighbors (kNN) Imputation method applied for performing symbolic regression(Al-Helali et al, 2018) Instance based Distribution-based nearest neighbors Applicable to high-dimensional data(Shah et al, 2017) Kernel based Adapt to many types of variables(Nebot-Troyano and Belanche-Muñoz, 2010) …”

mentioning

confidence: 99%

A systematic review of machine learning-based missing value imputation techniques

Thomas

Rajabi

2021

DTA

View full text Add to dashboard Cite

PurposeThe primary aim of this study is to review the studies from different dimensions including type of methods, experimentation setup and evaluation metrics used in the novel approaches proposed for data imputation, particularly in the machine learning (ML) area. This ultimately provides an understanding about how well the proposed framework is evaluated and what type and ratio of missingness are addressed in the proposals. The review questions in this study are (1) what are the ML-based imputation methods studied and proposed during 2010–2020? (2) How the experimentation setup, characteristics of data sets and missingness are employed in these studies? (3) What metrics were used for the evaluation of imputation method?Design/methodology/approachThe review process went through the standard identification, screening and selection process. The initial search on electronic databases for missing value imputation (MVI) based on ML algorithms returned a large number of papers totaling at 2,883. Most of the papers at this stage were not exactly an MVI technique relevant to this study. The literature reviews are first scanned in the title for relevancy, and 306 literature reviews were identified as appropriate. Upon reviewing the abstract text, 151 literature reviews that are not eligible for this study are dropped. This resulted in 155 research papers suitable for full-text review. From this, 117 papers are used in assessment of the review questions.FindingsThis study shows that clustering- and instance-based algorithms are the most proposed MVI methods. Percentage of correct prediction (PCP) and root mean square error (RMSE) are most used evaluation metrics in these studies. For experimentation, majority of the studies sourced the data sets from publicly available data set repositories. A common approach is that the complete data set is set as baseline to evaluate the effectiveness of imputation on the test data sets with artificially induced missingness. The data set size and missingness ratio varied across the experimentations, while missing datatype and mechanism are pertaining to the capability of imputation. Computational expense is a concern, and experimentation using large data sets appears to be a challenge.Originality/valueIt is understood from the review that there is no single universal solution to missing data problem. Variants of ML approaches work well with the missingness based on the characteristics of the data set. Most of the methods reviewed lack generalization with regard to applicability. Another concern related to applicability is the complexity of the formulation and implementation of the algorithm. Imputations based on k-nearest neighbors (kNN) and clustering algorithms which are simple and easy to implement make it popular across various domains.

show abstract

mentioning

confidence: 99%

A systematic review of machine learning-based missing value imputation techniques

Thomas

Rajabi

2021

DTA

View full text Add to dashboard Cite

show abstract

“…Secondly, some learning methods can directly deal on incomplete data without explicitly estimating the missing values such as C4.5 [191], fuzzy-based approach [26], and learn/sup ++ ensemble method [127]. The third approach is called imputation, where the missing values are firstly replaced by estimated values then the learning is carried out using the complete imputed data set [4,8]. Unlike classification, the investigation of symbolic regression on incomplete data has not received adequate efforts.…”

Section: List Of Publicationsmentioning

confidence: 99%

“…The first collection of data sets is shown in Table 3. 4 and more details can be found in the data repositories UCI [62] and OpenML [241]. To introduce synthetic incompleteness to the complete data sets, MAR missingness mechanism is used to impose missing values with 10%, 30%, and 50% missingness ratios.…”

Section: Real-world Data Sets With Synthetic Incompletenessmentioning

confidence: 99%

Genetic Programming for Symbolic Regression on Incomplete Data

Al-Helali¹

Self Cite

View full text Add to dashboard Cite

Symbolic regression is the process of constructing mathematical expressions that best fit given data sets, where a target variable is expressed in terms of input variables. Unlike traditional regression methods, which optimise the parameters of pre-defined models, symbolic regression learns both the model structure and its parameters simultaneously. Genetic programming (GP) is a biologically-inspired evolutionary algorithm, that automatically generates computer programs to solve a given task. The flexible representation of GP along with its ``white box" nature makes it a dominant method for symbolic regression. Moreover, GP has been successfully employed for different learning tasks such as feature selection and transfer learning. Data incompleteness is a pervasive problem in symbolic regression, and machine learning in general, especially when dealing with real-world data sets. One common approach to handling data missingness is data imputation. Data imputation is the process of estimating missing values based on existing data. Another approach to deal with incomplete data is to build learning algorithms that directly work with missing values. Although a number of methods have been proposed to tackle the data missingness issue in machine learning, most studies focus on classification tasks. Little attention has been paid to symbolic regression on incomplete data. The existing symbolic regression methods are only applicable when the given data set is complete. The overall goal of the thesis is to improve the performance of symbolic regression on incomplete data by using GP for data imputation, instance selection, feature selection, and transfer learning. This thesis develops an imputation method to handle missing values for symbolic regression. The method integrates the instance-based similarity of the k-nearest neighbour method with the feature-based predictability of GP to estimate the missing values. The results show that the proposed method outperforms existing popular imputation methods. This thesis develops an instance selection method for improving imputation for symbolic regression on incomplete data. The proposed method has the ability to simultaneously build imputation and symbolic regression models such that the performance is improved. The results show that involving instance selection with imputation advances the performance of using the imputation alone. High-dimensionality is a serious data challenge, which is even more difficult on incomplete data. To address this problem in symbolic regression tasks, this thesis develops a feature selection method that can select a good set of features directly from incomplete data. The method not only improves the regression accuracy, but also enhances the efficiency of symbolic regression on high-dimensional incomplete data. Another challenging problem is data shortage. This issue is even more challenging when the data is incomplete. To handle this situation, this thesis develops transfer learning methods to improve symbolic regression in domains with incomplete and limited data. These methods utilise two powerful abilities of GP: feature construction and feature selection. The results show the ability of these methods to achieve positive transfer learning from domains with complete data to different (but related) domains with incomplete data. In summary, the thesis develops a range of approaches to improving the effectiveness and efficiency of symbolic regression on incomplete data by developing a number of GP-based methods. The methods are evaluated using different types of data sets considering various missingness and learning scenarios.

show abstract

“…GP is derived from Genetic Algorithm in which the members of the algorithm's population are computer programs. Popularised by Koza, it utilises evolutionary operations to evolve computer programs [130,59] and has been used extensively for solving a vast variety of problems [249], [3], [16]. As is depicted in Algorithm 2.1, GP for evolving routing policies has four general steps:…”

Section: Genetic Programmingmentioning

confidence: 99%

“…Based on the attribute values of the tasks, we can see that the indices of the tasks selected by the routing policy in the three decision situations are 2, 1, and 3, respectively. This forms the phenotypic behaviour vector [2,1,3].…”

Section: Phenotypic Characterisation Of Routing Policiesmentioning

confidence: 99%

Transfer Optimisation in Genetic Programming for Solving Uncertain Capacitated Arc Routing Problem

Ardeh¹

View full text Add to dashboard Cite

Uncertain Capacitated Arc Routing Problem (UCARP) is a combinatorial optimisation problem with many important real-world applications. Genetic programming (GP) is a powerful machine learning technique that has been successfully used to automatically evolve routing policies for UCARP. Reusability is an open issue in the field of UCARP and in this direction, an open challenge is the case of scenario changes, e.g. change in the number of vehicles and probability distributions of random demands, which typically requires new training procedures to be initiated. Considering the expensive training cost of evolving routing policies for UCARP, a promising strategy is to learn and reuse knowledge from a previous problem-solving process to improve the effectiveness and efficiency of solving a new related problem, i.e. transfer learning. The overall goal of this thesis is to develop novel knowledge transfer algorithms for GP for solving UCARP to handle environment changes more effectively and efficiently. To fulfil this goal, a plethora of machine learning techniques, i.e. surrogate models, feature selection, searching and specialised genetic operators, are utilised in this thesis. First, this thesis explores the effectiveness of the existing transfer optimisation methods for solving UCARP. Accordingly, one of the main directions of this thesis is towards identifying the nature of transferable knowledge, which can impact the quality of knowledge transfer for GP to solve UCARP. For this purpose, a collection of the state-of-the-art transfer optimisation GP algorithms are evaluated for UCARP. After identifying some potential gaps in the literature, a number of preliminary transfer optimisation algorithms are proposed that supplement the literature. To evaluate the algorithms, a large set of knowledge transfer scenarios with various source and target problems were designed based on real-world datasets. According to the results, none of the methods showed significant improvement in the effectiveness of the trained UCARP routing policies. These results revealed the need for more effective transfer optimisation methods specifically designed for UCARP. Furthermore, our investigations revealed that the presence of duplicates in knowledge sources is one of the main challenges for effective transfer optimisation in solving UCARP. Second, we propose approaches to handling the presence of duplicates in the transferred knowledge. The first approach increases population diversity after knowledge transfer to counteract the loss of diversity that is introduced by the presence of duplicates in the transferred knowledge. In the second approach, the duplicates are removed from the transferred knowledge. Then, the transferred knowledge is utilised to create a diverse initial GP population of high-quality individuals. Both approaches are investigated through detailed experimental studies. The results indicate that, while the first approach did not perform better than GP with knowledge transfer, the second can improve the effectiveness of training routing policies with GP significantly. Third, this thesis proposes a novel algorithm that transfers the phenotypic characteristics of the routing policies for solving the source problem. In the new algorithm, the most fit and unique source routing policies are utilised for initialising GP for solving the target problem. Then, a tabu list is placed on the source routing policies and the GP process is prohibited from recreating any of the source routing policies. The motivation for this approach is that, due to the existence of similarity between the source and target problems, source routing policies are unlikely to have a good performance for the target problem. Our experimental studies confirmed that by prohibiting GP from recreating source policies, and the computational resources will be spent on searching and evaluating new regions of the search space, which can lead to discovering better solutions. Fourth, this thesis proposes a novel knowledge transfer algorithm based on the idea of maintaining the transferred knowledge as an auxiliary population. In this approach, first, the best individuals of the duplicate-free knowledge source are used to initialise GP. Additionally, these transferred individuals are also maintained as an auxiliary population and are evolved alongside the main population. To save the computational cost, the auxiliary population is evolved with a surrogate method. Additionally, an elaborate knowledge exchange mechanism between the two populations is devised that emphasises transferring high-quality and unique individuals, the transfer of which can improve the diversity of the receiving population. This allows GP to overcome the problem of losing its population diversity during the evolutionary process. Our detailed experimental results confirmed the superior performance of the proposed algorithm and confirmed that the proposed method improved the phenotypic diversity of GP population.

show abstract

A Hybrid GP-KNN Imputation for Symbolic Regression with Missing Values

Cited by 23 publications

References 12 publications

A systematic review of machine learning-based missing value imputation techniques

A systematic review of machine learning-based missing value imputation techniques

Genetic Programming for Symbolic Regression on Incomplete Data

Transfer Optimisation in Genetic Programming for Solving Uncertain Capacitated Arc Routing Problem

Contact Info

Product

Resources

About