Performance Improvement of Feature Selection Methods based on Bio-Inspired Algorithms

Abstract

Feature Selection is one of methods to improve the classification accuracy of data in the field of machine learning. Many feature selection algorithms have been proposed and discussed for years. However, the problem of finding the optimal feature subset from full data still remains to be a difficult problem. Bio-inspired algorithms are well-known evolutionary algorithms based on the principles of behavior of organisms, and very useful methods to find the optimal solution in optimization problems. Bio-inspired algorithms are also used in the field of feature selection problems. So in this paper we proposed new improved bio-inspired algorithms for feature selection. We used well-known bio-inspired algorithms, Genetic Algorithm (GA) and Particle Swarm Optimization (PSO), to find the optimal subset of features that shows the best performance in classification accuracy. In addition, we modified the bio-inspired algorithms considering the prior importance (prior relevance) of each feature. We chose the mRMR method, which can measure the goodness of single feature, to set the prior importance of each feature. We modified the evolution operators of GA and PSO by using the prior importance of each feature. We verified the performance of the proposed methods by experiment with datasets. Feature selection methods using GA and PSO produced better performances in terms of the classification accuracy. The modified method with the prior importance demonstrated improved performances in terms of the evolution speed and the classification accuracy.

Optimal Feature Selection of Speech using Particle Swarm Optimization Integrated with mRMR for Determining Human Emotion State

“…, where p(x,y) is the joint probability distribution function of X and Y, and marginal probability distribution functions of X and Y are p(x) and p(y) [4].Hence the mutual information quantifies the information shared by X and Y by measuring the reduction in uncertainty of one variable when the other variable is concentrated upon.…”

“…Hence a simple and integrated technique can be applied for the feature selection of speech. PSO+mRMR for feature subset selection gives good results for various fields [4] and since mRMR is classifier independent, PSO+mRMR can be used for Speech Feature Subset Selection which is subjected to GMM to resolve all problems. Selecting only the optimal features reduces the work load on GMM model for classifying the states.…”

“…Accuracy of the learning algorithm is defined by the fitness function. Binary string representation of the position vector of a particle urges to modify the traditional PSO operations accordingly [4]. The modified operations for PSO can be given as follows [4]: Let x ia (t) be the ath element of X i (t).Then x ia (t) sets its value to 0 and 1 iteratively as follows: …”

“…It reduces the complexity of mRMR and aims at selecting features with high significance and eliminates features with low significance. It is obtained by just altering eq(5) [4].In the combined algorithm , the relevance of a feature is checked to see in a subset D3 and D1 and the values are set to 1 or 0 correspondingly. PSO+mRMR technique have been used for feature selection in various applications such as Audiology, Dermatology, Image Segmentation and many others.…”

“…The mRMR technique redundancy of each feature is also considered by the mutual information of them. [4]Let y n be a feature in a feature subset S then its redundancy is defined by following equation:…”