Input feature selection by mutual information based on Parzen window

Abstract. This paper presents a supervised feature selection method applied to regression problems. The selection method uses a Dissimilarity matrix originally developed for classification problems, whose applicability is extended here to regression and built using the conditional mutual information between features with respect to a continuous relevant variable that represents the regression function. Applying an agglomerative hierarchical clustering technique, the algorithm selects a subset of the original set of features. The proposed technique is compared with other three methods. Experiments on four data-sets of different nature are presented to show the importance of the features selected from the point of view of the regression estimation error (using Support Vector Regression) considering the Root Mean Squared Error (RM SE).

show abstract

“…The starting values were fixed at: h 1,0 = h 2,0 = 1 2 log(N ) , as in [11], and the lower and upper bounds at…”

Section: Resultsmentioning

confidence: 99%

Feature Selection in Regression Tasks Using Conditional Mutual Information

Carmona

Sotoca

Pla

et al. 2011

Pattern Recognition and Image Analysis

View full text Add to dashboard Cite

show abstract

“…Hence, an estimation of the mutual information is required and different methods can be employed. Among the possible methods are histogram-based [25], kernel density estimation [26], knearest neighbour [27], Parzen window [28], B-spline [29], adaptive partitioning [30,31] and fuzzy-based [32] approaches. These estimation methods typically involve some pre-set parameters whose optimal values heavily depend on problem characteristics.…”

Section: Related Workmentioning

confidence: 99%

A new maximum relevance-minimum multicollinearity (MRmMC) method for feature selection and ranking

Senawi

Wei

Billings

2017

Pattern Recognition

View full text Add to dashboard Cite

A substantial amount of datasets stored for various applications are often high dimensional with redundant and irrelevant features. Processing and analyzing data under such circumstances is time consuming and makes it difficult to obtain efficient predictive models. There is a strong need to carry out analyses for high dimensional data in some lower dimensions, and one approach to achieve this is through feature selection. This paper presents a new relevancy-redundancy approach, called the maximum relevance-minimum multicollinearity (MRmMC) method, for feature selection and ranking, which can overcome some shortcomings of existing criteria. In the proposed method,

show abstract

“…Mutual information, which measures the mutual dependence of two variables, has historically been used successfully in the related area of feature selection, such as in [9], [10], and [11]. Typical feature selection algorithms involving mutual information select features with high mutual information with respect to a classification label or class, such that the features are relevant to the classification task.…”

Section: Mutual Information As Relevance Measurementioning

confidence: 99%

“…The technique described in [9] for computing the mutual information via approximation of equation 1 is used here. The Parzen windowing technique is used to model the distribution of the continuous-valued feature vectors:…”

Section: Mutual Information As Relevance Measurementioning

confidence: 99%

Towards Structured Approaches to Arbitrary Data Selection and Performance Prediction for Speaker Recognition

Lei

2009

Advances in Biometrics

View full text Add to dashboard Cite

Abstract. We developed measures relating feature vector distributions to speaker recognition (SR) performances for performance prediction and potential arbitrary data selection for SR. We examined the measures of mutual information, kurtosis, correlation, and measures pertaining to intraand inter-speaker variability. We applied the measures on feature vectors of phones to determine which measures gave good SR performance prediction of phones standalone and in combination. We found that mutual information had an -83.5% correlation with the Equal Error Rates (EERs) of each phone. Also, Pearson's correlation between the feature vectors of two phones had a -48.6% correlation with the relative EER improvement of the score-level combination of the phones. When implemented in our new data-selection scheme (which does not require a SR system to be run), the measures allowed us to select data with 2.13% overall EER improvement (on SRE08) over data selected via a brute-force approach, at a fifth of the computational costs.

show abstract

Input feature selection by mutual information based on Parzen window

Cited by 578 publications

References 11 publications

Feature Selection in Regression Tasks Using Conditional Mutual Information

Feature Selection in Regression Tasks Using Conditional Mutual Information

A new maximum relevance-minimum multicollinearity (MRmMC) method for feature selection and ranking

Towards Structured Approaches to Arbitrary Data Selection and Performance Prediction for Speaker Recognition

Contact Info

Product

Resources

About