A Group Feature Ranking and Selection Method Based on Dimension Reduction Technique in High-Dimensional Data

Abstract

Group feature selection methods select the important group features by removing the irrelevant group features for reducing the complexity of the model. There are a few group feature selection methods that used ranking techniques. Ranking methods provide the relative importance of each group. For this purpose, we developed a sparse group feature ranking method based on the dimension reduction technique for high dimensional data. Firstly, we applied relief to each group to remove irrelevant features. Secondly, we extract the new feature that represents each feature group. To this end, we reduce the dimension of the group feature into a single dimension by applying Fisher linear discriminant analysis (FDA) for each feature group. At last, we estimate the relative importance of the new feature by applying random forest and selecting those important features that have higher importance compared with other ones. In the end, different machinelearning algorithms were used to train and test the models. For the experiment, we compared the proposed with supervised group lasso (SGL) methods by using real-life high-dimensional datasets. Results show that the proposed method selects a few important group features just like the existing group feature selection method and provides the ranking and relative importance of all group features. SGL slightly performs better on logistic regression whereas the proposed method performs better on support vector machine, random forest, and gradient boosting in terms of classification performance metrics.

Spatiotemporal Patterns of Methane and Nitrous Oxide Emissions in China’s Inland Waters Identified by Machine Learning Technique

“…In order to predict GHG emission fluxes based on water quality data, we developed a RF model to describe their relationships. Before constructing the model, feature screening was performed to reduce the number of input features in order to improve the computational efficiency and reliability. − The features were sorted according to their importance scores obtained from each prediction, and the one with the lowest score was excluded for the subsequent round of prediction. The model construction process is illustrated in Figure S3.…”

“…To improve the modeling efficiency, feature selection was performed to exclude the features with relatively low importance scores. − In our study, we initially evaluated the prediction performance of the model using 8 input features and recorded the importance score for each feature. The feature importance score reflects the contribution of each feature to the model’s predictions.…”

A Review on the Classification of Partial Discharges in Medium-Voltage Cables: Detection, Feature Extraction, Artificial Intelligence-Based Classification, and Optimization Techniques

“…Relief is a pre-processing filter that removes irrelevant features by calculating feature weights using proxy statistics. The feature's relevance to the target variable is indicated by its feature score, which runs from −1 to +1 [115]. Nevertheless, it does not handle redundancy and is limited to two-class classification problems.…”

En-PaFlower: An Ensemble Approach using PSO and Flower Pollination Algorithm for Cancer Diagnosis

“…Gene expression data, on the other hand, include high dimensionalities that are unimportant while looking for diseases. We cannot trust the high-dimensional gene expression data since it contains redundant information and is thus useless [4]. The inability to process all data at once, as well as the possibility that its subset data processing may lead to the reasons behind over-fitting, information loss, and other such issues, can all be briefly described as limitations of microarray data that are directly affecting the accuracy of classification [5,6].…”