A new wrapper feature selection approach using neural network

Abstract

This paper presents a new feature selection (FS) algorithm based on the wrapper approach using neural networks (NNs). The vital aspect of this algorithm is the automatic determination of NN architectures during the FS process. Our algorithm uses a constructive approach involving correlation information in selecting features and determining NN architectures. We call this algorithm as constructive approach for FS (CAFS). The aim of using correlation information in CAFS is to encourage the search strategy for selecting less correlated (distinct) features if they enhance accuracy of NNs. Such an encouragement will reduce redundancy of information resulting in compact NN architectures. We evaluate the performance of CAFS on eight benchmark classification problems. The experimental results show the essence of CAFS in selecting features with compact NN architectures.

A Comparative Analysis of Machine Learning Algorithms to Build a Predictive Model for Detecting Diabetes Complications

“…Wrapper techniques depend on the classification algorithm, which is used to evaluate the subsets of features, but they are more expensive when it comes to computation [44]. Despite this weakness, they often provide better outcomes; wrappers are used widely in many applications [45].…”

Scalable Feature Selection in High-Dimensional Data Based on GRASP

“…They need to train a predictor (classifier or regression model) to evaluate each feature subset; therefore, they are more precise but slower than the filters. The main challenges in wrapper methods are selecting a proper predictor (Li et al 2009;Maldonado and Weber 2009;Monirul Kabir, Monirul Islam, and Murase 2010;Sánchez-Maroño and Alonso-Betanzos 2011) and how to generate appropriate subsets (Macas et al 2012;Tay and Cao 2001;Vignolo, Milone, and Scharcanski 2013). Finally, hybrid methods use both filter and wrapper evaluators simultaneously (Bermejo et al 2012;Bermejo, Gámez, and Puerta 2011;Gheyas and Smith 2010;Ruiz et al 2012).…”

Analytics-at-scale of Sensor Data for Digital Monitoring in Nuclear Plants (3^rd Annual Report)

“…However, wrappers are prone to overfitting and can be computationally expensive [10]. Hybrid and ensemble methods integrate filters and wrappers alike, thereby benefiting from their complementary approaches [10], [14]. Three of the most common feature selection techniques-apart from the ones used in this report-are mutual information (MI) [15], recursive feature elimination [16], and analysis of variance (ANOVA) tests.…”