Popular Ensemble Methods: An Empirical Study

Opitz, David W.; Maclin, Richard

doi:10.1613/jair.614

Cited by 2,347 publications

(1,272 citation statements)

References 37 publications

Supporting

Mentioning

1,176

Contrasting

Unclassified

Order By: Relevance

“…It can be seen that the TER may vary due to the variation of the testing data, even if the size of the data set remains the same. Therefore, standard 10-fold cross validations have been used for result presentation [7]- [12]. In the cross validation, initially available training patterns were partitioned into 10 equal or nearly equal sets, and for each turn, one set was reserved as a testing set, while the remaining nine sets were used for constructing DTs.…”

Section: A Benchmark Problems and General Experimental Methodologymentioning

confidence: 99%

“…A fixed number of 10 DTs were constructed for an ensemble, except for DECORATE. Previous studies have revealed that ensemble with this number of classifiers is able to show good generalization [7], [18], [19]. To be comparable to the other methods, the maximum number of DTs per ensemble in DECORATE was defined as 10 and the maximum number of trial DTs was 15.…”

Section: A Benchmark Problems and General Experimental Methodologymentioning

confidence: 99%

“…To overcome this, it produces a relatively large number of classifiers and selects a subset of classifiers for the final ensemble. The method seems complicated and its computational cost is relatively high due to several steps for pattern generation, relatively large number of classifiers creation and selection of classifiers [18].With the above discussed DT based ensemble methods, a number of ensemble methods have also been investigated with neural network (NN) as base classifier [7], [18]. Some NN based ensemble methods share similar techniques of DT ensembles and some other are only for NN, such as, negative correlation learning [18], [19].…”

mentioning

confidence: 99%

“…With the above discussed DT based ensemble methods, a number of ensemble methods have also been investigated with neural network (NN) as base classifier [7], [18]. Some NN based ensemble methods share similar techniques of DT ensembles and some other are only for NN, such as, negative correlation learning [18], [19].…”

mentioning

confidence: 99%

See 3 more Smart Citations

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Akhand¹,

Rahman²,

Murase³

2013

IJMLC

View full text Add to dashboard Cite

Abstract-An ensemble method produces diverse classifiers and combines their decisions for ensemble's decision. A number of methods have been investigated for constructing ensemble in which some of them train classifiers with the generated patterns. This study investigates a new technique of training pattern generation that is easy and effective for ensemble construction. The method modifies feature values of some patterns with the values of other patterns to generate different patterns for different classifiers. The ensemble of decision trees based on the proposed technique was evaluated using a suite of 30 benchmark classification problems, and was found to achieve performance better than or competitive with related conventional methods. Furthermore, two different hybrid ensemble methods have been investigated incorporating the proposed technique of pattern generation with two popular ensemble methods bagging and random subspace method (RSM). It is found that the performance of bagging and RSM algorithms can be improved by incorporating feature values modification with their training processes. Experimental investigation of different types of modification techniques finds that feature values modification with pattern values in the same class is better for generalization.Index Terms-Decision tree ensemble, diversity, feature values modification, generalization, pattern generation. I. INTRODUCTIONThe goal of ensemble construction with several classifiers is to achieve better generalization ability over individual classifiers. The inspiration for building an ensemble is the same as for establishing a committee of people: each member of the committee should be as competent as possible, but the members should be complementary to one another. If the members are not complementary, i.e., if they always agree, the committee is unnecessary as any one member could perform the task of the committee. If the members are complementary, then when one or a few members make an error, there is a high probability that the remaining members can correct his error. Thus, for ensemble construction, proper diversity among classifiers (also called base classifiers) is considered to be an important parameter so that the failure of one may be compensated by others [1], [2].An ensemble method produces diverse classifiers and combines their decisions for ensemble's decision. As a base M. M. Hafizur Rahman is with Dept. of Computer Science, KICT, International Islamic University Malaysia, Jalan Gombak, 50728 Selayang, Selangor, Malaysia (e-mail: hafizur@iium.edu.my).K. Murase is with Graduate School of Engineering, University of Fukui, 3-9-1 Bunkyo, Fukui 910-8507, Japan (e-mail: murase@u-fukui.ac.jp).classifier, decision trees (DTs) are one of the most commonly used methods because they are efficient [3], [4]. Considerable work has been done to determine the effective ways for constructing diverse DTs so that the benefit of ensemble construction could be achieved. There are many ways, such as using different training sets and learning metho...

show abstract

Section: A Benchmark Problems and General Experimental Methodologymentioning

confidence: 99%

Section: A Benchmark Problems and General Experimental Methodologymentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Akhand¹,

Rahman²,

Murase³

2013

IJMLC

View full text Add to dashboard Cite

show abstract

“…Many researchers have studied ensemble techniques and diversity measures. Hansen and Salamon have provided the theoretical basis on ensemble [5], while Opitz and Maclin have performed empirical ensemble experiments comprehensively [6]. Zhou et al have analyzed the effect on the number of participating classifiers into ensemble in both theoretical and empirical studies [7].…”

Section: Introductionmentioning

confidence: 99%

Cancer Prediction Using Diversity-Based Ensemble Genetic Programming

Hong

Cho

2005

Modeling Decisions for Artificial Intelligence

View full text Add to dashboard Cite

Abstract. Combining a set of classifiers has often been exploited to improve the classification performance. Accurate as well as diverse base classifiers are prerequisite to construct a good ensemble classifier. Therefore, estimating diversity among classifiers has been widely investigated. This paper presents an ensemble approach that combines a set of diverse rules obtained by genetic programming. Genetic programming generates interpretable classification rules, and diversity among them is directly estimated. Finally, several diverse rules are combined by a fusion method to generate a final decision. The proposed method has been applied to cancer classification using gene expression profiles, which is one of the important issues in bioinformatics. Experiments on several popular cancer datasets have demonstrated the usability of the method. High performance of the proposed method has been obtained, and the accuracy has increased by diversity among the base classification rules.

show abstract

Feasibility study of intelligent fault location estimation methods for double‐circuit transmission lines

Swetapadma

Chakrabarti

Abdelaziz

2021

Int Trans Electr Energ Syst

View full text Add to dashboard Cite

Fault location estimation methods have been suggested by researchers decades back after the development of transmission lines for power transmission. But there are several drawbacks exist such as incapability to locate high resistance faults, boundary faults, intercircuit faults, change in line parameters, evolving faults, etc. Hence, a method should be developed, which can address all the problems of conventional fault location estimation methods. One of the solutions can be artificial intelligence (AI) due to the ability to learn and think. In this article, an AI-based fault location estimation method has been suggested for transmission lines, which use the measurement from one end of the line. A comparative study of four AI methods has been carried out, and one suitable method has been chosen. Three methods are used for signal processing, and one suitable method has been chosen for signal processing and feature selection. The proposed AI-based method has error in location within 1 km for most fault cases, and maximum error in locating fault is 2.91 km. The performance enhancement of the proposed AI-based method over existing methods is that it does not require a communication link as it uses only one end signal; it can locate high resistance faults, boundary faults, intercircuit faults, unaffected by variation in line parameters, variation in power flow angle, etc. The proposed method can be implemented in real power network, which is feasible, easy, low cost, and robust.

show abstract

Popular Ensemble Methods: An Empirical Study

Cited by 2,347 publications

References 37 publications

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Pattern Generation through Feature Values Modification and Decision Tree Ensemble Construction

Cancer Prediction Using Diversity-Based Ensemble Genetic Programming

Feasibility study of intelligent fault location estimation methods for double‐circuit transmission lines

Contact Info

Product

Resources

About