Relevant and Redundant Feature Analysis with Ensemble Classification

Duangsoithong, Rakkrit; Windeatt, Terry

doi:10.1109/icapr.2009.36

Cited by 18 publications

(12 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Redundant features increase dimensionality unnecessarily [17], and worsen learning performance when facing the shortage of data. It is also shown empirically that removing redundant features can result in significant performance improvement [2], [7], [8], [46], [51]. Below, we propose a new framework for similarity preserving feature selection.…”

Section: Discussionmentioning

confidence: 98%

“…There are four image data: AR10P, 3 PIE10P, 4 PIX10P, 5 and ORL10P, 6 two Microarray data: TOX and CLL-SUB from the GEO gene expression data depository 7 with retrieval ID GDS1454 and GDS968, respectively, and two text data: RELATHE (BASEBALL versus HOCKEY) and PCMAC (PC versus MAC) from the 20-newgroup data. 8 The two text data sets are preprocessed by the TMG package [48] with standard processes. Detailed information of the data sets is listed in Table 1.…”

Section: Experimental Studymentioning

confidence: 99%

See 1 more Smart Citation

On Similarity Preserving Feature Selection

Zhao

Wang

Liu

et al. 2013

IEEE Trans. Knowl. Data Eng.

290

119

View full text Add to dashboard Cite

In the literature of feature selection, different criteria have been proposed to evaluate the goodness of features. In our investigation, we notice that a number of existing selection criteria implicitly select features that preserve sample similarity, and can be unified under a common framework. We further point out that any feature selection criteria covered by this framework cannot handle redundant features, a common drawback of these criteria. Motivated by these observations, we propose a new "Similarity Preserving Feature Selection" framework in an explicit and rigorous way. We show, through theoretical analysis, that the proposed framework not only encompasses many widely used feature selection criteria, but also naturally overcomes their common weakness in handling feature redundancy. In developing this new framework, we begin with a conventional combinatorial optimization formulation for similarity preserving feature selection, then extend it with a sparse multiple-output regression formulation to improve its efficiency and effectiveness. A set of three algorithms are devised to efficiently solve the proposed formulations, each of which has its own advantages in terms of computational complexity and selection performance. As exhibited by our extensive experimental study, the proposed framework achieves superior feature selection performance and attractive properties.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Experimental Studymentioning

confidence: 99%

On Similarity Preserving Feature Selection

Zhao

Wang

Liu

et al. 2013

IEEE Trans. Knowl. Data Eng.

290

119

View full text Add to dashboard Cite

show abstract

“…Several studies proposed the removal of redundant features as this might jeopardize prediction accuracy due to overfitting [28], while others noticed that the removal of this type of feature may cause the exclusion of potentially relevant features [29]. Most of existing works propose to find redundant features through correlations [30,31,28] or clustering similar patterns into feature clusters [32,33].…”

Section: Definitionsmentioning

confidence: 99%

A survey on feature drift adaptation: Definition, benchmark, challenges and future directions

Barddal

Gomes

Enembreck

et al. 2017

Journal of Systems and Software

View full text Add to dashboard Cite

Data stream mining is a fast growing research topic due to the ubiquity of data in several real-world problems. Given their ephemeral nature, data stream sources are expected to undergo changes in data distribution, a phenomenon called concept drift. This paper focuses on one specific type of drift that has not yet been thoroughly studied, namely feature drift. Feature drift occurs whenever a subset of features becomes, or ceases to be, relevant to the learning task, thus, learners must detect and adapt to these changes accordingly. We survey existing work on feature drift adaptation in both explicit and implicit approaches. Additionally, we benchmark several algorithms and a naive proposal in synthetic and real-world datasets. The results from our experiments indicate the need for future research in this area as even naive approaches produced gains in accuracy while reducing resources usage. Finally, we state current research topics, challenges and future directions for feature drift adaptation.

show abstract

“…It has been empirically shown that removing redundant features can result in significant performance improvement [69]. Some algorithms have been developed to handle redundancy in feature selection [69,40,56,210,6,43]. However, there is still not much systematical work that studies how to adapt the large number of existing algorithms (especially the algorithms based on the filter model) to handle redundant features.…”

Section: Redundant Featuresmentioning

confidence: 99%

Spectral Feature Selection for Data Mining

Zhao¹,

Liu²

2011

View full text Add to dashboard Cite

This book is for people interested in feature selection research. Feature selection is an essential technique for dimensionality reduction and relevance detection. In advanced data mining software packages, such as SAS Enterpriser Miner, SPSS Modeler, Weka, Spider, Orange, and scikits.learn, feature selection procedures are indispensable components for successful data mining applications. The rapid advance of computer-based high-throughput techniques provides unparalleled opportunities for humans to expand capabilities in production, services, communications, and research. Meanwhile, immense quantities of high-dimensional data keep on accumulating, thus challenging and stimulating the development of feature selection research in two major directions. One trend is to improve and expand the existing techniques to meet new challenges, and the other is to develop brand new techniques directly targeting the arising challenges. In this book, we introduce a novel feature selection technique, spectral feature selection, which forms a general platform for studying existing feature selection algorithms as well as developing novel algorithms for new problems arising from real-world applications. Spectral feature selection is a unified framework for supervised, unsupervised and semi-supervised feature selection. With its great generalizability, it includes many existing successful feature selection algorithms as its special cases, allowing the joint study of these algorithms to achieve better understanding and gain interesting insights. Based on spectral feature selection, families of novel feature selection algorithms can also be designed to address new challenges, such as handling feature redundancy, processing very large-scale data sets, and utilizing various types of knowledge to achieve multi-source feature selection. With the steady and speedy development of feature selection research, we sincerely hope that this book presents a distinctive contribution to feature selection research, and inspires new developments in feature selection. We have no doubt what feature selection can impact on the processing of massive, high-dimensional data with complex structure in the near future. We are truly optimistic that in another 10 years when we look back, we will be humbled by the accreted power of feature selection, and by its indelible contributions to machine learning, data mining, and many real-world applications.

show abstract

Relevant and Redundant Feature Analysis with Ensemble Classification

Cited by 18 publications

References 10 publications

On Similarity Preserving Feature Selection

On Similarity Preserving Feature Selection

A survey on feature drift adaptation: Definition, benchmark, challenges and future directions

Spectral Feature Selection for Data Mining

Contact Info

Product

Resources

About