Feature-subspace aggregating: ensembles for stable and unstable learners

Ting, Kai Ming; Wells, Jonathan R.; Tan, Swee Chuan; Teng, Shyh Wei; Webb, Geoffrey I.

doi:10.1007/s10994-010-5224-5

Cited by 40 publications

(23 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feating (Ting et al 2011) is a generic ensemble learning technique that also builds upon the ensembling strategy of AODE. Like AnDE, Feating operates by building a local model for each combination of n attribute values.…”

Section: Relationship To Featingmentioning

confidence: 99%

See 1 more Smart Citation

Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification

et al. 2011

Self Cite

View full text Add to dashboard Cite

Averaged n-Dependence Estimators (AnDE) is an approach to probabilistic classification learning that learns by extrapolation from marginal to full-multivariate probability distributions. It utilizes a single parameter that transforms the approach between a lowvariance high-bias learner (Naive Bayes) and a high-variance low-bias learner with Bayes optimal asymptotic error. It extends the underlying strategy of Averaged One-Dependence Estimators (AODE), which relaxes the Naive Bayes independence assumption while retaining many of Naive Bayes' desirable computational and theoretical properties. AnDE further relaxes the independence assumption by generalizing AODE to higher-levels of dependence. Extensive experimental evaluation shows that the bias-variance trade-off for Averaged 2-Dependence Estimators results in strong predictive accuracy over a wide range of data sets. It has training time linear with respect to the number of examples, learns in a single pass through the training data, supports incremental learning, handles directly missing values, and is robust in the face of noise. Beyond the practical utility of its lower-dimensional variants, AnDE is of interest in that it demonstrates that it is possible to create low-bias high-variance generative learners and suggests strategies for developing even more powerful classifiers.

show abstract

Section: Relationship To Featingmentioning

confidence: 99%

“…In Sect. 3 we discuss how the AnDE algorithms relate to Feating (Ting et al 2011), a generic approach to ensembling that also builds upon techniques pioneered by AODE. In Sect.…”

Section: Introductionmentioning

confidence: 99%

Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification

et al. 2011

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, a new ensemble technique, called feature-subspace aggregating (Feating) [24], was proposed that was shown to have nice performances. The key point of these ensemble methods is aggregating a large number of models built using sub-datasets randomly generated using for example bootstrapping.…”

Section: Model Population Analysis and Ensemble Learningmentioning

confidence: 99%

Model Population Analysis for Statistical Model Comparison

Li¹,

Liang²,

Xu³

2012

Chemometrics in Practical Applications

View full text Add to dashboard Cite

“…Near the end of the project we read a paper entitled Feature-subspace aggregating: ensembles for stable and unstable learners, [34]. Its core algorithm turned out to be inapplicable for sparse sequence data, but we were intrigued with the basic idea of trying to build CRFs that are expert in various "locations" in feature space.…”

Section: A15 Feature-class/label Transition Poolsmentioning

confidence: 99%

Network and Ensemble Enabled Entity Extraction in Informal Text (NEEEEIT) final report

Kegelmeyer

Shead

Dunlavy

2013

View full text Add to dashboard Cite

This SAND report summarizes the activities and outcomes of the Network and Ensemble Enabled Entity Extraction in Informal Text (NEEEEIT) LDRD project, which addressed improving the accuracy of conditional random fields for named entity recognition through the use of ensemble methods.Conditional random fields (CRFs) are powerful, flexible probabilistic graphical models often used in supervised machine learning prediction tasks associated with sequence data. Specifically, they are currently the best known option for named entity recognition (NER) in text. NER is the process of labeling words in sentences with semantic identifiers such as "person", "date", or "organization".Ensembles are a powerful statistical inference meta-method that can make most supervised machine learning methods more accurate, faster, or both. Ensemble methods are normally best suited to "unstable" classification methods with high variance error. CRFs applied to NER are very stable classifiers, and as such, would initially seem to be resistant to the benefits of ensembles.The NEEEEIT project nonetheless worked out how to generalize ensemble methods to CRFs, demonstrated that accuracy can indeed be improved by proper use of ensemble techniques, and generated a new CRF code, "pyCrust" and a surrounding application environment, "NEEEEIT", which implement those improvements.The summary practical advice that results from this work, then, is:• When making use of CRFs for label prediction tasks in machine learning, use the pyCrust CRF base classifier with NEEEEIT's bagging ensemble implementation. (If those codes are not available, then de-stablize your CRF code via every means available, and generate the bagged training sets by hand.)• If you have ample pre-processing computational time, do "forward feature selection" to find and remove counter-productive feature classes.

show abstract

Feature-subspace aggregating: ensembles for stable and unstable learners

Cited by 40 publications

References 23 publications

Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification

Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification

Model Population Analysis for Statistical Model Comparison

Network and Ensemble Enabled Entity Extraction in Informal Text (NEEEEIT) final report

Contact Info

Product

Resources

About