A Systematic Literature Review: Recursive Feature Elimination Algorithms

Abstract

Recursive feature elimination (RFE) is a feature selection algorithm that works by gradually eliminating unimportant features. RFE has become a popular method for feature selection in various machine learning applications, such as classification and prediction. However, there is no systematic literature review (SLR) that discusses recursive feature elimination algorithms. This article conducts a SLR on RFE algorithms. The goal is to provide an overview of the current state of the RFE algorithm. This SLR uses IEEE Xplore, ScienceDirect, Springer, and Scopus (publish and publish) databases from 2018 to 2023. This SLR received 76 relevant papers with 49% standard RFEs, 43% strategy RFEs, and 8% modified RFEs. Research using RFE continues to increase every year, from 2018 to 2023. The feature selection method used simultaneously or for comparison is based on a filter approach, namely Pearson correlation, and an embedded approach, namely random forest. The most widely used machine learning algorithms are support vector machines and random forests, with 19.5% and 16.7%, respectively. Strategy RFE and modified RFE can be referred to as hybrid RFEs. Based on relevant papers, it is found that the RFE strategy is broadly divided into two categories: using RFE after other feature selection methods and using RFE simultaneously with other methods. Modification of the RFE is done by modifying the flow of the RFE. The modification process is divided into two categories: before the process of calculating the smallest weight criteria and after calculating the smallest weight criteria. Calculating the smallest weight criteria in this RFE modification is still a challenge at this time to obtain optimal results.

Developing a Model to Predict Self-Reported Student Performance during Online Education Based on the Acoustic Environment

“…The RFE is adaptable to any type of algorithm, rendering it a versatile method that is applicable across various scenarios. The Recurse RFE method is classified by some authors as a wrapper method [45]. Some other sources consider it a wrapper-type feature selection method that also uses filter-based feature selection internally [46].…”

Effectiveness of Variable Selection Methods for Machine Learning and Classical Statistical Models

“…Within algorithmic modelling, we can also use one of the wrapper methods, recursive feature elimination (RFE). In this method, smaller and smaller subsets of variables are considered, the least important variables are removed from the current set based on a measure of importance, until a predetermined number of variables is reached [Kohavi, John 1997;Priyatno et al 2024]. The approach can however be computationally expensive.…”