Dynamic Trees for Learning and Design

Taddy, Matthew A.; Gramacy, Robert B.; Polson, Nicholas G.

doi:10.1198/jasa.2011.ap09769

Cited by 91 publications

(117 citation statements)

References 25 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…While Markov chain Monte Carlo (MCMC) is the frequently used, we show in Section 4 that the widely available implementations of BCART generally perform poorly compared to recent particle filtering algorithms (Lakshminarayanan et al, 2013;Taddy et al, 2011). We review the MCMC methods used to fit BCART before discussing more recent methods based on sequential Monte Carlo.…”

Section: Computational Detailsmentioning

confidence: 99%

“…There are several possibilities for choosing q(T (t−1) → T (t) ), but Lakshminarayanan et al (2013) recommend simply using π(T (t−1) → T (t) ) and choosing M large. Taddy et al (2011) take a different approach, directly using a dynamic model…”

Section: Sequential Monte Carlomentioning

confidence: 99%

“…Because the predictor is onedimensional, it allows for easy visual comparisons of methods. We consider the BCART model given in (2.1), the BART model (2.3), as well as the dynamic regression trees model of Taddy et al (2011). We also consider the smoothed variant SBART of BART described in Section 2.2, the random forests algorithm of Breiman (2001) as implemented in the grf package, and the tree boosting algorithm in the package xgboost.…”

Section: Nonparametric Regression and Classificationmentioning

confidence: 99%

See 2 more Smart Citations

A review of tree-based Bayesian methods

Linero

2017

CSAM

View full text Add to dashboard Cite

Tree-based regression and classification ensembles form a standard part of the data-science toolkit. Many commonly used methods take an algorithmic view, proposing greedy methods for constructing decision trees; examples include the classification and regression trees algorithm, boosted decision trees, and random forests. Recent history has seen a surge of interest in Bayesian techniques for constructing decision tree ensembles, with these methods frequently outperforming their algorithmic counterparts. The goal of this article is to survey the landscape surrounding Bayesian decision tree methods, and to discuss recent modeling and computational developments. We provide connections between Bayesian tree-based methods and existing machine learning techniques, and outline several recent theoretical developments establishing frequentist consistency and rates of convergence for the posterior distribution. The methodology we present is applicable for a wide variety of statistical tasks including regression, classification, modeling of count data, and many others. We illustrate the methodology on both simulated and real datasets.

show abstract

Section: Computational Detailsmentioning

confidence: 99%

Section: Sequential Monte Carlomentioning

confidence: 99%

Section: Nonparametric Regression and Classificationmentioning

confidence: 99%

See 1 more Smart Citation

A review of tree-based Bayesian methods

Linero

2017

CSAM

View full text Add to dashboard Cite

show abstract

“…This was pioneered by [1] in the case of dynamic trees, where the tree structure is viewed as the latent state that evolves dynamically. In effect, a "particle cloud" of dynamic trees are employed to track parsimonious regression and classification surfaces as data arrive sequentially.…”

Section: Introductionmentioning

confidence: 99%

Information-Theoretic Data Discarding for Dynamic Trees on Data Streams

Anagnostopoulos

Gramacy

2013

Entropy

View full text Add to dashboard Cite

Ubiquitous automated data collection at an unprecedented scale is making available streaming, real-time information flows in a wide variety of settings, transforming both science and industry. Learning algorithms deployed in such contexts often rely on single-pass inference, where the data history is never revisited. Learning may also need to be temporally adaptive to remain up-to-date against unforeseen changes in the data generating mechanism. Online Bayesian inference remains challenged by such transient, evolving data streams. Nonparametric modeling techniques can prove particularly ill-suited, as the complexity of the model is allowed to increase with the sample size. In this work, we take steps to overcome these challenges by porting information theoretic heuristics, such as exponential forgetting and active learning, into a fully Bayesian framework. We showcase our methods by augmenting a modern non-parametric modeling framework, dynamic trees, and illustrate its performance on a number of practical examples. The end product is a powerful streaming regression and classification tool, whose performance compares favorably to the state-of-the-art.

show abstract

“…Full details on the model can be obtained from the article by Taddy et al [51]. The brief overview of how it works is as follows.…”

Section: Dynamic Treesmentioning

confidence: 99%

Minimizing the cost of iterative compilation with active learning

William

Petoumenos

Wang

et al. 2017

2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

View full text Add to dashboard Cite

Since performance is not portable between platforms, engineers must fine-tune heuristics for each processor in turn. This is such a laborious task that high-profile compilers, supporting many architectures, cannot keep up with hardware innovation and are actually out-of-date. Iterative compilation driven by machine learning has been shown to be efficient at generating portable optimization models automatically. However, good quality models require costly, repetitive, and extensive training which greatly hinders the wide adoption of this powerful technique.In this work, we show that much of this cost is spent collecting training data, runtime measurements for different optimization decisions, which contribute little to the final heuristic. Current implementations evaluate randomly chosen, often redundant, training examples a pre-configured, almost always excessive, number of times -a large source of wasted effort. Our approach optimizes not only the selection of training examples but also the number of samples per example, independently. To evaluate, we construct 11 high-quality models which use a combination of optimization settings to predict the runtime of benchmarks from the SPAPT suite. Our novel, broadly applicable, methodology is able to reduce the training overhead by up to 26x compared to an approach with a fixed number of sample runs, transforming what is potentially months of work into days.

show abstract

Dynamic Trees for Learning and Design

Cited by 91 publications

References 25 publications

A review of tree-based Bayesian methods

A review of tree-based Bayesian methods

Information-Theoretic Data Discarding for Dynamic Trees on Data Streams

Minimizing the cost of iterative compilation with active learning

Contact Info

Product

Resources

About