SIP-FS: a novel feature selection for data representation

Abstract

Multiple features are widely used to characterize real-world datasets. It is desirable to select leading features with stability and interpretability from a set of distinct features for a comprehensive data description. However, most of existing feature selection methods focus on the predictability (e.g., prediction accuracy) of selected results yet neglect stability. To obtain compact data representation, a novel feature selection method is proposed to improve stability, and interpretability without sacrificing predictability (SIP-FS). Instead of mutual information, generalized correlation is adopted in minimal redundancy maximal relevance to measure the relation between different feature types. Several feature types (each contains a certain number of features) can then be selected and evaluated quantitatively to determine what types contribute to a specific class, thereby enhancing the so-called interpretability of features. Moreover, stability is introduced in the criterion of SIP-FS to obtain consistent results of ranking. We conduct experiments on three publicly available datasets using one-versus-all strategy to select class-specific features. The experiments illustrate that SIP-FS achieves significant performance improvements in terms of stability and interpretability with desirable prediction accuracy and indicates advantages over several state-of-the-art approaches.

An Approach for Optimal Feature Selection in Machine Learning using Global Sensitivity Analysis

“…4. The last type of feature selection is called embedded methods in Fig 4, a compromise that includes the selection of functions for model learning between the filtering and the wrapping methods [10]. Therefore, these approaches are worthy of wrappers and filter (1) because they need to interact with the classification algorithm; and (2) because they do not require an iterative evaluation of the functional sets.…”