Comparing Swarm Intelligence Algorithms for Dimension Reduction in Machine Learning

Abstract

Nowadays, the high-dimensionality of data causes a variety of problems in machine learning. It is necessary to reduce the feature number by selecting only the most relevant of them. Different approaches called Feature Selection are used for this task. In this paper, we propose a Feature Selection method that uses Swarm Intelligence techniques. Swarm Intelligence algorithms perform optimization by searching for optimal points in the search space. We show the usability of these techniques for solving Feature Selection and compare the performance of five major swarm algorithms: Particle Swarm Optimization, Artificial Bee Colony, Invasive Weed Optimization, Bat Algorithm, and Grey Wolf Optimizer. The accuracy of a decision tree classifier was used to evaluate the algorithms. It turned out that the dimension of the data can be reduced about two times without a loss in accuracy. Moreover, the accuracy increased when abandoning redundant features. Based on our experiments GWO turned out to be the best. It has the highest ranking on different datasets, and its average iteration number to find the best solution is 30.8. ABC obtained the lowest ranking on high-dimensional datasets.

Enhancing EEG-Based Mental Stress State Recognition Using an Improved Hybrid Feature Selection Algorithm

“…Depending on the dataset, the exploration and exploitation variables of PSO should be adjusted independently. This is consistent with another study, which found that typical PSOs have unbalanced exploration and exploitation and could be improved by adjusting its parameters [68]. Among the SI algorithms, the bat algorithm (BAT), grey wolf optimizer (GWO), moth optimization algorithm (MVO), and whale optimization algorithm (WOA) performed the best in all factors: enhancing accuracy and lowering feature number while maintaining an acceptable execution time.…”

“…Swarm intelligence (SI) has quickly evolved in recent years and relatively provide an efficient solution for tackling NP-hard computational problems, such as high-dimensionality features [3,68]. FS is also seen as an optimization issue, with methods aiming to select a subset of important features that balances accuracy while minimizing the number of features required.…”

“…Only one firefly moves randomly according to chaotic search, limiting its global searchability, which researchers addressed by proposing several modifications to improve global searchability [69]. According to [68], GWO is superior to PSO and BAT algorithms. This is because BAT takes longer to discover the optimal solution, whilst PSO requires extra parameter tweaking to achieve the best result.…”

Implementation of Features Selection Based on Dragonfly Optimization Algorithm

“…It involves moving agents, modifying their characteristics, and updating the better solution (s). The global better solution is eventually returned Swarm optimization techniques, like other search algorithms, require an appropriate balance of exploration and exploitation [2].…”

“…Among the many well-known SI algorithms, Dragonfly optimization was selected and a hard-voting classifier was employed to evaluate the feature subsets. [2].…”