A Multi-objective Feature Selection Approach Based on Binary PSO and Rough Set Theory

Abstract

Abstract. Feature selection has two main objectives of maximising the classification performance and minimising the number of features. However, most existing feature selection algorithms are single objective wrapper approaches. In this work, we propose a multi-objective filter feature selection algorithm based on binary particle swarm optimisation (PSO) and probabilistic rough set theory. The proposed algorithm is compared with other five feature selection methods, including three PSO based single objective methods and two traditional methods. Three classification algorithms (naïve bayes, decision trees and k-nearest neighbours) are used to test the generality of the proposed filter algorithm. Experiments have been conducted on six datasets of varying difficulty. Experimental results show that the proposed algorithm can automatically evolve a set of non-dominated feature subsets. In almost all cases, the proposed algorithm outperforms the other five algorithms in terms of both the number of features and the classification performance (evaluated by all the three classification algorithms). This paper presents the first study on using PSO and rough set theory for multi-objective feature selection.

Simultaneous feature selection and weighting – An evolutionary multi-objective optimization approach

“…Instead, an optimal number of features are required to obtain maximum accuracy in classification for most datasets. A very similar work was done by the Cervante et al (2013), where they used binary PSO scheme to optimize the classification performance and number of features selected. Recently, Xue et al (2014) proposed a Multi Objective Differential Evolution based FS (DEMOFS) technique, which minimizes the classification error rate and the number of features selected.…”

A binary ABC algorithm based on advanced similarity scheme for feature selection

“…Evaluation criterion: An evaluation criterion is expected to measure the quality of a feature subset accurately and inexpensively. Fundamentally, all evaluation criteria are based on the either classification performance or the characteristics of the data itself [39,40]. 4.…”

Hybrid ACO-PSO Based Approaches for Feature Selection

“…Two multi-objective FS algorithms are developed by applying mutual information and entropy as two different filter evaluation criteria in the proposed framework. Cervante et al [47] propose a novel feature selection algorithm based on a multi-objective PSO and probabilistic rough set theory with the goal of obtaining a set of non-dominated features subsets.…”