Ensemble Feature Learning of Genomic Data Using Support Vector Machine

Abstract

The identification of a subset of genes having the ability to capture the necessary information to distinguish classes of patients is crucial in bioinformatics applications. Ensemble and bagging methods have been shown to work effectively in the process of gene selection and classification. Testament to that is random forest which combines random decision trees with bagging to improve overall feature selection and classification accuracy. Surprisingly, the adoption of these methods in support vector machines has only recently received attention but mostly on classification not gene selection. This paper introduces an ensemble SVM-Recursive Feature Elimination (ESVM-RFE) for gene selection that follows the concepts of ensemble and bagging used in random forest but adopts the backward elimination strategy which is the rationale of RFE algorithm. The rationale behind this is, building ensemble SVM models using randomly drawn bootstrap samples from the training set, will produce different feature rankings which will be subsequently aggregated as one feature ranking. As a result, the decision for elimination of features is based upon the ranking of multiple SVM models instead of choosing one particular model. Moreover, this approach will address the problem of imbalanced datasets by constructing a nearly balanced bootstrap sample. Our experiments show that ESVM-RFE for gene selection substantially increased the classification performance on five microarray datasets compared to state-of-the-art methods. Experiments on the childhood leukaemia dataset show that an average 9% better accuracy is achieved by ESVM-RFE over SVM-RFE, and 5% over random forest based approach. The selected genes by the ESVM-RFE algorithm were further explored with Singular Value Decomposition (SVD) which reveals significant clusters with the selected data.

Emerging trend of big data analytics in bioinformatics: a literature review

“…Haller et al 2012Gene expression classification using SVM Vanitha et al (2015) Ortholog detection in yeast species using imbalanced classification approaches including SVM Galpert et al (2015) Evolutionary feature selection using SVM and other techniques using MapReduce Peralta et al (2015) Genomic feature learning using SVMrecursive feature elimination algorithm Anaissi et al (2016) Decision trees Employing decision tree learning for processing of large datasets Hall et al (1998) RainForest, a framework supporting construction of fast decision tree for classification of large datasets Johannes Gehrke et al 2000Predictive decision tree model for analysing racial disparities in breast cancer Palit et al (2009) A streaming parallel decision tree algorithm for classification of largescale datasets and streaming data…”

Sparse Feature Learning Using Ensemble Model for Highly-Correlated High-Dimensional Data

“…Recent work, called the fuzzy forests method, has been proposed by [6] which uses recursive feature elimination random forests to select the features from the correlated feature blocks. Fuzzy forests depends on random forest feature selection which has a high computational complexity in terms of running time compared to the feature selection method using the support vector machine [2]. Furthermore, the fuzzy forests method does not take into account the imbalanced data problem which may generate features which are biased towards the majority class.…”

“…The ESVM-RFE [2] ranks the features by constructing an ensemble of SVM models in each iteration of SVM-RFE using a random bootstrap subset from the training set. Then, it aggregates all the feature rankings as an ensemble vote.…”

“…Recently, we proposed an ensemble SVM (ESVM-RFE) algorithm [2] for individual feature ranking in high-dimensional data. The ESVM-RFE uses the ensemble strategy with the SVM [4] classifier as the base learning model.…”

“…Each module contains the correlated features, and the correlation between the modules is low. Second, our previous study of the ESVM-RFE algorithm [2] is used to select the most important features within each module. Finally, the selected features are aggregated and again, ESVM-RFE is applied to select the optimal features of the modules.…”