A General Wrapper Approach to Selection of Class-Dependent Features

Wang, Lipo; Zhou, Nina; Feng, Chunhua

doi:10.1109/tnn.2008.2000395

Cited by 113 publications

(43 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second, and more promising approach, consists of using a wrapper method [47] coupled with an evolutionary instance selection method such as the one described in Algorithm 5. Similar methods have been used to train neural networks [38], [48], although without focusing on boosting the classifier.…”

Section: Constructing Ensembles Using Other Classifiersmentioning

confidence: 99%

Constructing Ensembles of Classifiers by Means of Weighted Instance Selection

García-Pedrajas

2009

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

In this paper, we approach the problem of constructing ensembles of classifiers from the point of view of instance selection. Instance selection is aimed at obtaining a subset of the instances available for training capable of achieving, at least, the same performance as the whole training set. In this way, instance selection algorithms try to keep the performance of the classifiers while reducing the number of instances in the training set. Meanwhile, boosting methods construct an ensemble of classifiers iteratively focusing each new member on the most difficult instances by means of a biased distribution of the training instances. In this work, we show how these two methodologies can be combined advantageously. We can use instance selection algorithms for boosting using as objective to optimize the training error weighted by the biased distribution of the instances given by the boosting method. Our method can be considered as boosting by instance selection. Instance selection has mostly been developed and used for k -nearest neighbor ( k -NN) classifiers. So, as a first step, our methodology is suited to construct ensembles of k -NN classifiers. Constructing ensembles of classifiers by means of instance selection has the important feature of reducing the space complexity of the final ensemble as only a subset of the instances is selected for each classifier. However, the methodology is not restricted to k-NN classifier. Other classifiers, such as decision trees and support vector machines (SVMs), may also benefit from a smaller training set, as they produce simpler classifiers if an instance selection algorithm is performed before training. In the experimental section, we show that the proposed approach is able to produce better and simpler ensembles than random subspace method (RSM) method for k-NN and standard ensemble methods for C4.5 and SVMs.

show abstract

Section: Constructing Ensembles Using Other Classifiersmentioning

confidence: 99%

Constructing Ensembles of Classifiers by Means of Weighted Instance Selection

García-Pedrajas

2009

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

show abstract

“…In the feature selection, we choose those representative vectors that can effectively capture the difference between the samples from different classes (in other words, they contain sufficient discriminative information) from three MPCA models. There are a great number of feature selection algorithms in the literature that can be roughly grouped into three categories: 1) filters [42]; 2) wrappers [44]; and 3) hybrid algorithms [45]. The filters usually select the features without needing the classifier performance, whereas the wrappers need the classifier performance to select the features.…”

Section: ) Motivation Of Feature Selection Scheme In Mpcamentioning

confidence: 99%

Modified Principal Component Analysis: An Integration of Multiple Similarity Subspace Models

Fan

Xu²,

Zuo

et al. 2014

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

We modify the conventional principal component analysis (PCA) and propose a novel subspace learning framework, modified PCA (MPCA), using multiple similarity measurements. MPCA computes three similarity matrices exploiting the similarity measurements: 1) mutual information; 2) angle information; and 3) Gaussian kernel similarity. We employ the eigenvectors of similarity matrices to produce new subspaces, referred to as similarity subspaces. A new integrated similarity subspace is then generated using a novel feature selection approach. This approach needs to construct a kind of vector set, termed weak machine cell (WMC), which contains an appropriate number of the eigenvectors spanning the similarity subspaces. Combining the wrapper method and the forward selection scheme, MPCA selects a WMC at a time that has a powerful discriminative capability to classify samples. MPCA is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality. MPCA outperforms the other state-of-the-art PCA-based methods in terms of both classification accuracy and clustering result. In addition, MPCA can be applied to face image reconstruction. MPCA can use other types of similarity measurements. Extensive experiments on many popular real-world data sets, such as face databases, show that MPCA achieves desirable classification results, as well as has a powerful capability to represent data.

show abstract

“…Most algorithms use either a sequential search (for example, [4,5,24,26,30]) or a global search (e.g., [11,23,[31][32][33][34][35]). On the basis of guiding the search strategies and evaluating the subsets, in contrast, the existing FS algorithms can be grouped into the following three approaches: wrapper (e.g., [4,6,30,[36][37][38]), filter (e.g., [40,41]), and hybrid (e.g., [23,42]). It is well-known that wrapper approaches always return features with a higher saliency than filter approaches, as the former utilize the association of features collectively during the learning process, but are computationally more expensive [2]).…”

Section: Existing Work For Feature Selectionmentioning

confidence: 99%

“…In the FS process, feature removal operations are performed sequentially, especially for those features that do not degrade accuracy of the NN upon removal. A class-dependent FS algorithm in [38], selects a desirable feature subset for each class. It first divides a C class classification problem into C two-class classification problems.…”

Section: Existing Work For Feature Selectionmentioning

confidence: 99%