Effective Discriminative Feature Selection With Nontrivial Solution

Hong, Tao; Hou, Chenping; Nie, Feiping; Jiao, Yuling; Yi, Dongyun

doi:10.1109/tnnls.2015.2424721

Cited by 154 publications

(52 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a given Fmeasure r, the corresponding cost matrix C is fixed and thus W is the only variable in Eq. (14). Thus, we develop an iterative algorithm to solve this problem.…”

Section: B Optimization Methodsmentioning

confidence: 99%

“…Algorithm 1 An iterative algorithm to solve the optimization problem in Eq. (14). Input: feature matrix X ∈ R d×n , label matrix Y ∈ R n×m and discretized F-measure value r. Output: projection matrix W ∈ R d×m .…”

Section: Convergence Analysismentioning

confidence: 99%

“…According to [21], the objective value of Eq. (14) monotonically decreases in iterations, which implies that Algorithm 1 can converge to a local minimum.…”

Section: Convergence Analysismentioning

confidence: 99%

See 2 more Smart Citations

Cost-Sensitive Feature Selection by Optimizing F-Measures

Luo

et al. 2018

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Feature selection is beneficial for improving the performance of general machine learning tasks by extracting an informative subset from the high-dimensional features. Conventional feature selection methods usually ignore the class imbalance problem, thus the selected features will be biased towards the majority class. Considering that F-measure is a more reasonable performance measure than accuracy for imbalanced data, this paper presents an effective feature selection algorithm that explores the class imbalance issue by optimizing F-measures. Since F-measure optimization can be decomposed into a series of cost-sensitive classification problems, we investigate the cost-sensitive feature selection by generating and assigning different costs to each class with rigorous theory guidance. After solving a series of cost-sensitive feature selection problems, features corresponding to the best F-measure will be selected. In this way, the selected features will fully represent the properties of all classes. Experimental results on popular benchmarks and challenging real-world data sets demonstrate the significance of cost-sensitive feature selection for the imbalanced data setting and validate the effectiveness of the proposed method.

show abstract

“…For a given Fmeasure r, the corresponding cost matrix C is fixed and thus W is the only variable in Eq. (14). Thus, we develop an iterative algorithm to solve this problem.…”

Section: B Optimization Methodsmentioning

confidence: 99%

Section: Convergence Analysismentioning

confidence: 99%

See 1 more Smart Citation

Cost-Sensitive Feature Selection by Optimizing F-Measures

Luo

et al. 2018

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…Existing feature selection algorithm can be categorized as supervised feature selection (on data with full class labels) [5]- [9], unsupervised feature selection (on data without class labels) [10]- [15], and semisupervised feature selection (on data with partial labels) [14], [16], [17]. Feature selection in unsupervised context is considered to be more difficult than the other two cases, since there is no target information available for training.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous Bayesian Clustering and Feature Selection Through Student’s ${t}$ Mixtures Model

Sun

Zhou

Keates

et al. 2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Abstract-In this paper, we proposed a generative model for feature selection under the unsupervised learning context. The model assumes that data are independently and identically sampled from a finite mixture of Student's t distributions, which can reduce the sensitiveness to outliers. Latent random variables that represent the features' salience are included in the model for the indication of the relevance of features. As a result, the model is expected to simultaneously realize clustering, feature selection, and outlier detection. Inference is carried out by a tree-structured variational Bayes algorithm. Full Bayesian treatment is adopted in the model to realize automatic model selection. Controlled experimental studies showed that the developed model is capable of modeling the data set with outliers accurately. Furthermore, experiment results showed that the developed algorithm compares favorably against existing unsupervised probability model-based Bayesian feature selection algorithms on artificial and real data sets. Moreover, the application of the developed algorithm on real leukemia gene expression data indicated that it is able to identify the discriminating genes successfully.

show abstract

“…One of the prime requirements for effective classification is the selection of relevant features adept at capturing internal structure of the data [3]. In addition, an effective feature set should display high inter-class diversity and adequate robustness to variations such as illumination effects, rotation and translation.…”

Section: Introductionmentioning

confidence: 99%

Discriminative Autoencoder for Feature Extraction: Application to Character Recognition

Gogna

Majumdar

2018

Neural Process Lett

View full text Add to dashboard Cite

Conventionally, autoencoders are unsupervised representation learning tools. In this work, we propose a novel discriminative autoencoder. Use of supervised discriminative learning ensures that the learned representation is robust to variations commonly encountered in image datasets. Using the basic discriminating autoencoder as a unit, we build a stacked architecture aimed at extracting relevant representation from the training data. The efficiency of our feature extraction algorithm ensures a high classification accuracy with even simple classification schemes like KNN (K-nearest neighbor). We demonstrate the superiority of our model for representation learning by conducting experiments on standard datasets for character/image recognition and subsequent comparison with existing supervised deep architectures like class sparse stacked autoencoder and discriminative deep belief network.The basic building blocks of these deep architectures are either the stochastic RBM's (Restricted Boltzmann Machines) [16] or the deterministic Autoencoders [17]. Given a training dataset, RBM tries to learn the network weights such that the similarity between the projection (of the training data) and the learned representation is maximized. Autoencoders (AE) on the other hand consists of two networks. The first one maps the input (training data) to the representation / feature space; the second network maps the representation space to the output (training data). Thus, an AE approximates an Identity operator; which may sound trivial, but by constraining the nodes or connections of the networks one can learn interesting representations of the data.RBM and AE are shallow architectures. Proponents of deep learning believe that better (compact / abstract) representation can be learnt by going deeper. However, learning the network weights for several layers is a difficult task. Usually, there is not enough data, the network overfits and loses its generalization ability thereby yielding subpar results at operational stage. In [17], authors presented a greedy mechanism to train the multilayer (stacked) architectures wherein each of the layer is individually trained to yield best possible representation which in turn acts as input to subsequent layer. Greedy approach learns only one network at a time, it has fewer parameters to learn, so even with limited training data, it yields better results during operation.

show abstract

Effective Discriminative Feature Selection With Nontrivial Solution

Cited by 154 publications

References 23 publications

Cost-Sensitive Feature Selection by Optimizing F-Measures

Cost-Sensitive Feature Selection by Optimizing F-Measures

Simultaneous Bayesian Clustering and Feature Selection Through Student’s ${t}$ Mixtures Model

Discriminative Autoencoder for Feature Extraction: Application to Character Recognition

Contact Info

Product

Resources

About