A Hybrid Swarm Intelligence Based Feature Selection Algorithm for High Dimensional Datasets

Abstract

High dimensional datasets expose a critical obstacle in machine learning. Feature selection overcomes this obstacle by eliminating duplicated and unimportant features from the dataset to increase the robustness of learning algorithms. This paper introduces a binary version of a hybrid swarm intelligence approach as a wrapper method for feature selection that gathers between the strengths of both the grey wolf and particle swarm optimizers. This approach is named Improved Binary Grey Wolf Optimization (IBGWO). The original version of this hybrid approach was proposed in the literature with a continuous search space as a high-level hybrid form, which runs the optimizers one after the other. Two different types of transfer functions, named S-Shaped and V-Shaped, are applied in this work to turn continuous data into binary. Nine of high-dimensional small-instance medical datasets are employed to assess the proposed approach. The experimental results demonstrate that IBGWO based on S-Shaped (IBGWO-S) outperforms the binary particle swarm and the binary grey wolf optimizers on six out of nine datasets according to the classification accuracy and fitness values. IBGWO-S selects the fewest features on 100% of the datasets. The results show IBGWO based on V-Shaped (IBGWO-V) outperforms the binary particle swarm and binary grey wolf optimizers on five datasets based on the classification accuracy and fitness values. The results indicate that IBGWO-V outperforms IBGWO-S in terms of all studied evaluation metrics. The results also show that IBGWO-S and IBGWO-V outperform eight metaheuristics known in the literature in selecting the relevant features with acceptable classification accuracy.

Feature selection based on chi-square and ant colony optimization for multi-label classification

“…The wrapper approach is popular in FS, where ML algorithms evaluate the interaction between features and the efficiency of different feature subsets. The wrapper uses a search strategy to select a subset of features that optimizes the performance of a given ML algorithm [9], [10].…”

“…SI is an algorithm for optimization problems that imitate animal behavior and natural life. Examples of SI are ant colony optimization (ACO) [13], particle swarm optimization (PSO) [14], cuckoo optimization algorithm (COA) [15], grey wolf optimizer (GWO) [9], [16], and more. Marco Dorigo presented the ACO algorithm based on the behavior of ants in the early 1990s [17].…”