Diversity in search strategies for ensemble feature selection

Tsymbal, Alexey; Pechenizkiy, Mykola; Cunningham, Pádraig

doi:10.1016/j.inffus.2004.04.003

Cited by 285 publications

(150 citation statements)

References 25 publications

Supporting

Mentioning

145

Contrasting

Unclassified

Order By: Relevance

“…To measure diversity, we employed the most commonly used pairwise plain disagreement measure technique [25]. The plain disagreement diversity for a networks pair i and j is given by…”

Section: E Experimental Analysismentioning

confidence: 99%

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Akhand¹,

Rahman²,

Murase³

2013

IJMLC

View full text Add to dashboard Cite

Abstract-An ensemble method produces diverse classifiers and combines their decisions for ensemble's decision. A number of methods have been investigated for constructing ensemble in which some of them train classifiers with the generated patterns. This study investigates a new technique of training pattern generation that is easy and effective for ensemble construction. The method modifies feature values of some patterns with the values of other patterns to generate different patterns for different classifiers. The ensemble of decision trees based on the proposed technique was evaluated using a suite of 30 benchmark classification problems, and was found to achieve performance better than or competitive with related conventional methods. Furthermore, two different hybrid ensemble methods have been investigated incorporating the proposed technique of pattern generation with two popular ensemble methods bagging and random subspace method (RSM). It is found that the performance of bagging and RSM algorithms can be improved by incorporating feature values modification with their training processes. Experimental investigation of different types of modification techniques finds that feature values modification with pattern values in the same class is better for generalization.Index Terms-Decision tree ensemble, diversity, feature values modification, generalization, pattern generation. I. INTRODUCTIONThe goal of ensemble construction with several classifiers is to achieve better generalization ability over individual classifiers. The inspiration for building an ensemble is the same as for establishing a committee of people: each member of the committee should be as competent as possible, but the members should be complementary to one another. If the members are not complementary, i.e., if they always agree, the committee is unnecessary as any one member could perform the task of the committee. If the members are complementary, then when one or a few members make an error, there is a high probability that the remaining members can correct his error. Thus, for ensemble construction, proper diversity among classifiers (also called base classifiers) is considered to be an important parameter so that the failure of one may be compensated by others [1], [2].An ensemble method produces diverse classifiers and combines their decisions for ensemble's decision. As a base M. M. Hafizur Rahman is with Dept. of Computer Science, KICT, International Islamic University Malaysia, Jalan Gombak, 50728 Selayang, Selangor, Malaysia (e-mail: hafizur@iium.edu.my).K. Murase is with Graduate School of Engineering, University of Fukui, 3-9-1 Bunkyo, Fukui 910-8507, Japan (e-mail: murase@u-fukui.ac.jp).classifier, decision trees (DTs) are one of the most commonly used methods because they are efficient [3], [4]. Considerable work has been done to determine the effective ways for constructing diverse DTs so that the benefit of ensemble construction could be achieved. There are many ways, such as using different training sets and learning metho...

show abstract

“…To measure diversity, we employed the most commonly used pairwise plain disagreement measure technique [25]. The plain disagreement diversity for a networks pair i and j is given by…”

Section: E Experimental Analysismentioning

confidence: 99%

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Akhand¹,

Rahman²,

Murase³

2013

IJMLC

View full text Add to dashboard Cite

show abstract

“…Feature selection plays a key role in many of them where each classifier is derived by considering a different subset of the original features [27,37]. Random subspace [20], where each feature subset is randomly generated, is one of the most representative methods of this kind.…”

Section: Related Work On Mcssmentioning

confidence: 99%

“…It seems that obtaining a high diversity between classifiers is the goal to be reached, when aiming to achieve performance improvement of MCSs. In the last few years, a group of researchers devoted their attention to the diversity measures [25,26,27]. Two measures can be highlighted from the large amount of proposals in this group: the difficulty (θ ) and the double fault (δ ):…”

Section: The Four Used Evaluation Criteria For Two-objective Nsga-iimentioning

confidence: 99%

“…In the current work we integrate FURIA-based fuzzy MCSs within the OCS strategy. Since there are many optimization criteria considered for MCS design such as accuracy, complexity, and diversity measures [1,25,26,27], the use of a EMO algorithm came naturally to our mind.…”

Section: Introductionmentioning

confidence: 99%

“…To do so, we consider the state-of-the-art NSGA-II algorithm [28] to perform classifier selection. Introducing diversity and complexity measures combined with error measures is an interesting approach, which has led to promising results in the area [12,25,26,27,29,30]. Hence, we have embedded three measures of this kind in the objective space of the fitness function combining them with an accuracy index, which resulted in five different bicriteria fitness functions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Study on the Use of Multiobjective Genetic Algorithms for Classifier Selection in FURIA-based Fuzzy Multiclassifiers

Trawiński¹,

Cordón²,

Quirin³

2012

IJCIS

View full text Add to dashboard Cite

In a preceding contribution, we conducted a study considering a fuzzy multiclassifier system (MCS) design framework based on Fuzzy Unordered Rule Induction Algorithm (FURIA). It served as the fuzzy rule classification learning algorithm to derive the component classifiers considering bagging and feature selection. In this work, we integrate this approach under the overproduce-and-choose strategy. A state-ofthe-art evolutionary multiobjective algorithm, namely NSGA-II, is used to provide a component classifier selection and improve FURIA-based fuzzy MCS. We propose five different fitness functions based on three different optimization criteria, accuracy, complexity, and diversity. Twenty UCI high dimensional datasets were considered in order to conduct the experiments. A combination between accuracy and diversity criteria provided very promising results, becoming competitive with classical MCS learning methods.

show abstract

Stability of Feature Selection Algorithms and Ensemble Feature Selection Methods in Bioinformatics

Yang

Zhou

Yang

et al. 2013

Biological Knowledge Discovery Handbook

View full text Add to dashboard Cite

Diversity in search strategies for ensemble feature selection

Cited by 285 publications

References 25 publications

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

A Study on the Use of Multiobjective Genetic Algorithms for Classifier Selection in FURIA-based Fuzzy Multiclassifiers

Stability of Feature Selection Algorithms and Ensemble Feature Selection Methods in Bioinformatics

Contact Info

Product

Resources

About