Mutual information-based method for selecting informative feature sets

Herman, Gunawan; Zhang, Bang; Wang, Yang; Ye, Getian; Chen, Fang

doi:10.1016/j.patcog.2013.04.021

Cited by 60 publications

(42 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly to some previous research [12,13,9], for each feature set size, we employed a linear support vector machine (with the regularization parameter set to 1) to obtain the 10-fold cross-validation error rate (or leave-one-out validation error if the data set contains less than 100 instances). Additionally, the same statistics [5] are also collected from two other classifiers, namely Naive Bayes (NB) and kNN classifier (k = 3).…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…To determine which normalization form performs better overall, following Herman et al [13], the one-sided paired t-test at 5% significance level was used to compare Form-1 and Form-2 with the baseline Form-0. The experiment results of SVM are shown in Table 2 where we use '+'/'−'/'=' to indicate that Form-0 performs 'better'/'worse'/'equally well' compared to the two other forms.…”

Section: Normalizationmentioning

confidence: 99%

See 1 more Smart Citation

Can high-order dependencies improve mutual information based feature selection?

Vinh

Zhou

Chan

et al. 2016

Pattern Recognition

View full text Add to dashboard Cite

Mutual information (MI) based approaches are a popular paradigm for feature selection. Most previous methods have made use of low-dimensional MI quantities that are only effective at detecting low-order dependencies between variables. Several works have considered the use of higher dimensional mutual information, but the theoretical underpinning of these approaches is not yet comprehensive. To fill this gap, in this paper, we systematically investigate the issues of employing high-order dependencies for mutual information based feature selection. We first identify a set of assumptions under which the original high-dimensional mutual information based criterion can be decomposed into a set of low-dimensional MI quantities. By relaxing these assumptions, we arrive at a principled approach for constructing higher dimensional MI based feature selection methods that takes into account higher order feature interactions. Our extensive experimental evaluation on real data sets provides concrete evidence that methodological inclusion of highorder dependencies improve MI based feature selection.

show abstract

Section: Experimental Evaluationmentioning

confidence: 99%

Section: Normalizationmentioning

confidence: 99%

Can high-order dependencies improve mutual information based feature selection?

Vinh

Zhou

Chan

et al. 2016

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Therefore, it is important to consider interdependencies between features and remove those which are redundant. Herman et al [5] provide an in-depth review of redundant feature selection using MI. In their paper, a general framework is proposed for the several techniques in defining redundancy and combining it with relevancy.…”

Section: Redundant Feature Selectionmentioning

confidence: 99%

“…In their paper, a general framework is proposed for the several techniques in defining redundancy and combining it with relevancy. One such method is minimal redundancy maximum relevance (mRMR) [1,11], referred to as MI difference in [5]. In mRMR, the relevancy, Rel(F, C), of a feature set, F , is given by the mean MI of the member features and the class labels, C, namely,…”

Section: Redundant Feature Selectionmentioning

confidence: 99%

See 1 more Smart Citation

Redundant Feature Selection for Telemetry Data

Taylor

Griffths

Bhalerao

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Feature sets in many domains often contain many irrelevant and redundant features, both of which have a negative effect on the performance and complexity of agents that use the data [8]. Supervised feature selection aims to overcome this problem by selecting features that are highly related to the class labels, yet unrelated to each other. One proposed technique to select good features with few inter-dependencies is minimal Redundancy Maximal Relevance (mRMR) [11], but this can be impractical with large feature sets. In many situations, features are extracted from signal data such as vehicle telemetry, medical sensors, or financial time-series, and it is possible for feature redundancies to exist both between features extracted from the same signal (intra-signal), and between features extracted from different signals (inter-signal). We propose a two stage selection process to take advantage of these different types of redundancy, considering intra-signal and inter-signal redundancies separately. We illustrate the process on vehicle telemetry signal data collected in a driver distraction monitoring project. We evaluate it using several machine learning algorithms: Random Forest; Naïve Bayes; and C4.5 Decision Tree. Our results show that this two stage process significantly reduces the computation required because of inter-dependency calculations, while having minimal detrimental effect on the performance of the feature sets produced.

show abstract

Uni- and Multi-Dimensional Clustering Via Bayesian Networks

Keivani

Peña

2016

Unsupervised Learning Algorithms

View full text Add to dashboard Cite

Mutual information-based method for selecting informative feature sets

Cited by 60 publications

References 14 publications

Can high-order dependencies improve mutual information based feature selection?

Can high-order dependencies improve mutual information based feature selection?

Redundant Feature Selection for Telemetry Data

Uni- and Multi-Dimensional Clustering Via Bayesian Networks

Contact Info

Product

Resources

About