Bi-Directional Feature Fixation-Based Particle Swarm Optimization for Large-Scale Feature Selection

Abstract

Feature selection, which aims to improve the classification accuracy and reduce the size of the selected feature subset, is an important but challenging optimization problem in data mining. Particle swarm optimization (PSO) has shown promising performance in tackling feature selection problems, but still faces challenges in dealing with large-scale feature selection in big data environment because of the large search space. Hence, this paper proposes a bi-directional feature fixation (BDFF) framework for PSO and provides a novel idea to reduce the search space in large-scale feature selection. BDFF uses two opposite search directions to guide particles to adequately search for feature subsets with different sizes. Based on the two different search directions, BDFF can fix the selection states of some features and then focus on the others when updating particles, thus narrowing the large search space. Besides, a self-adaptive strategy is designed to help the swarm concentrate on a more promising direction for search in different stages of evolution and achieve a balance between exploration and exploitation. Experimental results on 12 widely-used public datasets show that BDFF can improve the performance of PSO on large-scale feature selection and obtain smaller feature subsets with higher classification accuracy.

Feature Selection Problem and Metaheuristics: A Systematic Literature Review about Its Formulation, Evaluation and Applications

“…Within this category, eight different functions were detected: (a) Accuracy: Defined in detail in Section 4.2.1 and mathematically in Equation ( 22). This objective function was pursued in [21,[27][28][29][30][31][32][33][34][35][36][37][38][39][40][41].…”

“…The authors report the average number of features selected in the performed runs and their standard deviation. This metric has been used in [21,27,28,32,34,[36][37][38]40,42,[44][45][46][47][48][49][50][51][53][54][55]57,58,[63][64][65][66][67][68][69][70][71][72]75,77,78,[81][82][83][84][85][86][87][88][89][90][91][92][93][94][95]…”

“…Particle swarm optimization is a population metaheuristic based on the swarming behavior of animals such as birds or fish [194]. This metaheuristic is the most used by the authors in [22,27,28,30,32,35,36,38,39,45,54,65,69,77,79,80,83,87,100,127,138,153,155,161,168,174,186]. The second most used metaheuristic is the grey wolf optimizer, a population metaheuristic based on the hunting behavior of grey wolves [195] and used in [51,71,84,87,88,95,101,105,128,134,137,157,160,170,173].…”

“…From the data presented in Figure 32, a predominant trend emerges related to the utilization of benchmark datasets [27][28][29][30][31][32][33][34][36][37][38][39][40][41][42][43][46][47][48][49][50][53][54][55][56][57][58][59][60][62][63][64][65][66][67][68][69][70][71][72][73][74][75][77][78][79][80][81][82][142]…”

“…Miscellaneous repositories: Several repositories mentioned in two articles were Ke Chen-Ph.D. Candidate Datasets Repository [36,38], which caters to specific academic projects; the UNB CIC [34,56] and UNSW Repositories [22,56], known for cybersecurity and network datasets; and the Mulan Library [41,63], emphasizing multi-label learning datasets. While these top ten repositories encompassed the majority of citations, a range of other repositories were also sourced, albeit less frequently.…”