Advancing feature ranking with hybrid feature ranking weighted majority model: a weighted majority voting strategy enhanced by the Harris hawks optimizer

Abstract

Feature selection (FS) is vital in improving the performance of machine learning (ML) algorithms. Despite its importance, identifying the most important features remains challenging, highlighting the need for advanced optimization techniques. In this study, we propose a novel hybrid feature ranking technique called the Hybrid Feature Ranking Weighted Majority Model (HFRWM2). HFRWM2 combines ML models with the Harris Hawks Optimizer (HHO) metaheuristic. HHO is known for its versatility in addressing various optimization challenges, thanks to its ability to handle continuous, discrete, and combinatorial optimization problems. It achieves a balance between exploration and exploitation by mimicking the cooperative hunting behavior of Harris’s hawks, thus thoroughly exploring the search space and converging towards optimal solutions. Our approach operates in two phases. First, an odd number of ML models, in conjunction with HHO, generate feature encodings along with performance metrics. These encodings are then weighted based on their metrics and vertically aggregated. This process produces feature rankings, facilitating the extraction of the top-K features. The motivation behind our research is twofold: to enhance the precision of ML algorithms through optimized FS and to improve the overall efficiency of predictive models. To evaluate the effectiveness of HFRWM2, we conducted rigorous tests on two datasets: “Australian” and “Fertility.” Our findings demonstrate the effectiveness of HFRWM2 in navigating the search space and identifying optimal solutions. We compared HFRWM2 against twelve other feature ranking techniques and found it to outperform them. This superiority was particularly evident in the graphical comparison of the “Australian” dataset, where HFRWM2 showed significant advancements in feature ranking.