Classification Acceleration via Merging Decision Trees

Fan, Chenglin; Li, Ping

doi:10.1145/3412815.3416886

Cited by 14 publications

(22 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As summarized in a recent paper on trees (Fan and Li, 2020), there are two recent implementation tricks that have made boosted tree models considerably more practical:…”

Section: Boosted Tree Modelsmentioning

confidence: 99%

Integrity Authentication in Tree Models

Zhao¹,

Lao²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Tree models are very widely used in practice of machine learning and data mining. In this paper, we study the problem of model integrity authentication in tree models. In general, the task of model integrity authentication is the design & implementation of mechanisms for checking/detecting whether the model deployed for the end-users has been tampered with or compromised, e.g., malicious modifications on the model. We propose an authentication framework that enables the model builders/distributors to embed a signature to the tree model and authenticate the existence of the signature by only making a small number of black-box queries to the model. To the best of our knowledge, this is the first study of signature embedding on tree models. Our proposed method simply locates a collection of leaves and modifies their prediction values, which does not require any training/testing data nor any re-training. The experiments on a large number of public classification datasets confirm that the proposed signature embedding process has a high success rate while only introducing a minimal prediction accuracy loss.

show abstract

“…As summarized in a recent paper on trees (Fan and Li, 2020), there are two recent implementation tricks that have made boosted tree models considerably more practical:…”

Section: Boosted Tree Modelsmentioning

confidence: 99%

Integrity Authentication in Tree Models

Zhao¹,

Lao²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The above developments were summarized in a recent paper on trees (Fan and Li, 2020). Readers are also referred to some interesting discussions in 2010 https://hunch.net/?p=1467.…”

Section: Introductionmentioning

confidence: 99%

Fast ABC-Boost: A Unified Framework for Selecting the Base Class in Multi-Class Classification

Li¹,

Zhao²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The work of Li (2009) in ICML'09 showed that the derivatives of the classical multi-class logistic regression loss function could be re-written in terms of a pre-chosen "base class" and Li (2009) applied the new derivatives in the popular boosting framework. In order to make use of the new derivatives, one must have a strategy to identify/choose the base class at each boosting iteration. The idea of "adaptive base class boost" (ABC-Boost) initially published in ICML'09, adopted a computationally expensive "exhaustive search" strategy for the base class at each iteration. It has been well demonstrated that ABC-Boost, when integrated with trees, can achieve substantial improvements in many multi-class classification tasks. Furthermore, Li (2010a) in UAI'10 derived the explicit second-order tree split gain formula which typically improved the classification accuracy considerably, compared with using only the fist-order information for tree-splitting, for both multi-class and binary-class classification tasks. Nevertheless, deploying ABC-Boost as published in Li (2009Li ( , 2010a has encountered severe computational difficulties due to its high computational cost of the exhaustive search strategy for the base class.

show abstract

“…As always, we should first solute to pioneers in boosting and trees, e.g., Brieman et al (1983); Schapire (1990); Freund (1995); Freund and Schapire (1997); Bartlett et al (1998); Schapire and Singer (1999); Friedman et al (2000); Friedman (2001). As summarized in a recent paper on merging decision trees (Fan and Li, 2020), in the past 15 years or so, multiple practical developments have enhanced the performance as well as the efficiency of boosted tree algorithms, including • The explicit (and robust) formula for tree-split criterion using the second-order gain information (Li, 2010b) (i.e., "Robust LogitBoost") typically improves the accuracy, compared to the implementation based on the criterion of using only the first-order gain information (Friedman, 2001). It is nowadays the standard implementation in popular tree platforms.…”

Section: Introductionmentioning

confidence: 99%