Learn Smart with Less: Building Better Online Decision Trees with Fewer Training Examples

Das, Aparna; Wang, Jin; Gandhi, Sahil M.; Lee, Jae; Wang, Wei; Zaniolo, Carlo

doi:10.24963/ijcai.2019/306

Cited by 10 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the research [12], Ariyam Das et al presented a memory-efficient bootstrap simulation heuristic (Mem-ES) that effectively accelerates the learning process. Experiments show that performing resampling techniques efficiently speeds up node splits for online decision tree learning.…”

Section: Related Workmentioning

confidence: 99%

Upgraded Very Fast Decision Tree: Energy Conservative Algorithm for Data Stream Classification

Lefa¹,

Abd-Elkader²,

Salem³

2023

IJACSA

View full text Add to dashboard Cite

Traditional machine learning (ML) techniques model knowledge using static datasets. With the increased use of the Internet in today's digital world, a massive amount of data is generated at an accelerated rate that must be handled. This data must be handled as soon as it arrives because it is continuous, and cannot be kept for a long period of time. Various methods exist for mining data from streams. When developing methods like these, the machine learning community put accuracy and execution time first. Numerous sorts of studies take energy consumption into consideration while evaluating data mining methods. However, this work concentrates on Very Fast Decision Tree, which is the most often used technique in data flow classification, despite the fact that it wastes a huge amount of energy on trivial calculations. The research presents a proposed mechanism for upgrading the algorithm's energy usage and restricts computational resources, without compromising the algorithm's efficiency. The mechanism has two stages: the first is to eliminate a set of bad features that increase computational complexity and waste energy, and the second is to group the good features into a candidate group that will be used instead of using all of the attributes in the next iteration. Experiments were conducted on real-world benchmark and synthetic datasets to compare the proposed method to state-of-the-art algorithms in previous works. The proposed algorithm works considerably better and faster with less energy while maintaining accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

Upgraded Very Fast Decision Tree: Energy Conservative Algorithm for Data Stream Classification

Lefa¹,

Abd-Elkader²,

Salem³

2023

IJACSA

View full text Add to dashboard Cite

show abstract

“…Except [4], the only existing dynamic algorithms for decision trees are incremental: they receive a stream of labeled examples and maintain a decision tree that performs well compared to the tree built on the examples seen so far. The first such algorithms were Hoeffding trees [9], which spurred a line of research on trees that adapt to so-called concept drifts [14,11,18,7,24,12,16,21]; see [17] for a survey. Unfortunately, all those algorithms assume the examples are i.i.d., which allows them to compute splits that have nearly-maximum gain with high confidence.…”

Section: Related Workmentioning

confidence: 99%

Fully-Dynamic Approximate Decision Trees With Worst-Case Update Time Guarantees

Bressan¹,

Sozio²

2023

Preprint

View full text Add to dashboard Cite

We give the first algorithm that maintains an approximate decision tree over an arbitrary sequence of insertions and deletions of examples, with strong guarantees on the worst-case running time per update request. For instance, we show how to maintain a decision tree where every vertex has Gini gain within an additive α of the optimum by performing O d log 4 n α 3 elementary operations per update, where d is the number of features and n the maximum size of the active set (the net result of the update requests). We give similar bounds for the information gain and the variance gain. In fact, all these bounds are corollaries of a more general result stated in terms of decision rules -functions that, given a set S of examples, decide whether to split S or predict a label. Decision rules give a unified view of greedy decision tree algorithms regardless of the example and label domains, and lead to a general notion of ǫ-approximate decision trees that, for natural decision rules such as those used by ID3 or C4.5, implies the gain approximation guarantees above. The heart of our work provides a deterministic algorithm that, given any decision rule and any ǫ > 0, maintains an ǫ-approximate tree using O d f (n) n poly h ǫ operations per update, where f (n) is the complexity of evaluating the rule over a set of n examples and h is the maximum height of the maintained tree.

show abstract

“…Several lines of research focus on different aspects of the tree learning problem -examples include shallow and sparse trees [Kumar et al, 2017], high-capacity decision nodes [Kumar et al, 2017, Balestriero, 2017 and/or leaf nodes [Hsieh et al, 2014], efficient inference [Jose et al, 2013], soft decisions [Frosst & Hinton, 2017], joint representation learning and tree learning [Kontschieder et al, 2015], and incremental tree learning in streaming scenarios [Domingos & Hulten, 2000, Jin & Agrawal, 2003, Manapragada et al, 2018, Das et al, 2019 (see Remark 1).…”

Section: Related Workmentioning

confidence: 99%

“…Remark 1 (Bandit vs standard online setting). The popular line of fast online/incremental tree induction/learning methods [Domingos & Hulten, 2000, Jin & Agrawal, 2003, Manapragada et al, 2018, Das et al, 2019 work in the setting where (a) labeled examples, i.e. (x i , y i ) arrive online, and (b) loss is implicit or known.…”

Section: Problem Setupmentioning

confidence: 99%

Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent

Karthikeyan¹,

Jain²,

Natarajan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Decision trees provide a rich family of highly non-linear but efficient models, due to which they continue to be the go-to family of predictive models by practitioners across domains. But learning trees is a challenging problem due to their highly discrete and non-differentiable decision boundaries. The state-of-the-art techniques use greedy methods that exploit the discrete tree structure but are tailored to specific problem settings (say, categorical vs real-valued predictions). In this work, we propose a reformulation of the tree learning problem that provides better conditioned gradients, and leverages successful deep network learning techniques like overparameterization and straight-through estimators. Our reformulation admits an efficient and accurate gradient-based algorithm that allows us to deploy our solution in disparate tree learning settings like supervised batch learning and online bandit feedback based learning.Using extensive validation on standard benchmarks, we observe that in the supervised learning setting, our general method is competitive to, and in some cases more accurate than, existing methods that are designed specifically for the supervised settings. In contrast, for bandit settings, where most of the existing techniques are not applicable, our models are still accurate and significantly outperform the applicable state-of-the-art methods.

show abstract

Learn Smart with Less: Building Better Online Decision Trees with Fewer Training Examples

Cited by 10 publications

References 15 publications

Upgraded Very Fast Decision Tree: Energy Conservative Algorithm for Data Stream Classification

Upgraded Very Fast Decision Tree: Energy Conservative Algorithm for Data Stream Classification

Fully-Dynamic Approximate Decision Trees With Worst-Case Update Time Guarantees

Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent

Contact Info

Product

Resources

About