Dynamic Classifier Chain with Random Decision Trees

Kulessa, Moritz; Mencía, Eneldo Loza

doi:10.1007/978-3-030-01771-2_3

Cited by 5 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kulessa and Loza Mencía [5] propose to integrate dynamic classifier chains in random decision trees (RDT) [20]. In contrast to the common induction of decision trees or to random forests, RDTs are constructed completely at random without following any predictive quality criterion.…”

Section: (Dynamic) Classifier Chainsmentioning

confidence: 99%

“…In contrast to the common induction of decision trees or to random forests, RDTs are constructed completely at random without following any predictive quality criterion. Kulessa and Loza Mencía [5] place tests on the labels at the inner nodes, which they can turn on and off without altering the original target of the RDT since it is only specified during prediction by the way of combing the statistics in the leaves. Hence, it is possible to simulate any binary base classifier of a CC in any possible chain sequence.…”

Section: (Dynamic) Classifier Chainsmentioning

confidence: 99%

“…More specifically, G j,v = i∈Iv g j,i and H j,v = i∈Iv h j,i extend to the labels 1 ≤ j ≤ N . In consequence, the objective (4) and gain functions (5) have to be adapted to consider gradient and hessian values from all classes. A common approach in multi-variate regression and multitarget classification is to compute the average loss of the model over all targets [18].…”

Section: Multi-label Xgboostmentioning

confidence: 99%

“…Dynamic chain approaches address the problem of finding a good sequence for a particular instance. For instance, Kulessa and Loza Mencía [5] propose to build an ensemble of random decision trees (RDT) with special label tests at the inner nodes. The approach predicts at each iteration the label for which the RDT is most certain and re-uses that information in subsequent iterations.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains

2020

Self Cite

View full text Add to dashboard Cite

Classifier chains is a key technique in multi-label classification, since it allows to consider label dependencies effectively. However, the classifiers are aligned according to a static order of the labels. In the concept of dynamic classifier chains (DCC) the label ordering is chosen for each prediction dynamically depending on the respective instance at hand. We combine this concept with the boosting of extreme gradient boosted trees (XGBoost), an effective and scalable state-of-the-art technique, and incorporate DCC in a fast multi-label extension of XGBoost which we make publicly available. As only positive labels have to be predicted and these are usually only few, the training costs can be further substantially reduced. Moreover, as experiments on eleven datasets show, the length of the chain allows for more control over the usage of previous predictions and hence over the measure one wants to optimize.

show abstract

Section: (Dynamic) Classifier Chainsmentioning

confidence: 99%

Section: (Dynamic) Classifier Chainsmentioning

confidence: 99%

Section: Multi-label Xgboostmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…This paper is based on (Kulessa and Loza Mencía, 2018) and (Bohlender et al, 2020). It provides an expanded, unified and definite description of these works, puts a stronger emphasis on the DCC framework, a more complete discussion of related work, and more detailed as well as several new experimental results.…”

Section: Introductionmentioning

confidence: 99%

Tree-Based Dynamic Classifier Chains

Mencía¹,

Kulessa²,

Bohlender³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Classifier chains are an effective technique for modeling label dependencies in multi-label classification. However, the method requires a fixed, static order of the labels. While in theory, any order is sufficient, in practice, this order has a substantial impact on the quality of the final prediction. Dynamic classifier chains denote the idea that for each instance to classify, the order in which the labels are predicted is dynamically chosen. The complexity of a naïve implementation of such an approach is prohibitive, because it would require to train a sequence of classifiers for every possible permutation of the labels. To tackle this problem efficiently, we propose a new approach based on random decision trees which can dynamically select the label ordering for each prediction. We show empirically that a dynamic selection of the next label improves over the use of a static ordering under an otherwise unchanged random decision tree model. In addition, we also demonstrate an alternative approach based on extreme gradient boosted trees, which allows for a more target-oriented training of dynamic classifier chains. Our results show that this variant outperforms random decision trees and other tree-based multi-label classification methods. More importantly, the dynamic selection strategy allows to considerably speed up training and prediction.

show abstract

Algorithm for Double-Layer Structure Multi-label Classification with Optimal Sequence Based on Attention Mechanism

Liu¹,

Tan²

2021

Simulation Tools and Techniques

View full text Add to dashboard Cite

Dynamic Classifier Chain with Random Decision Trees

Cited by 5 publications

References 18 publications

Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains

Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains

Tree-Based Dynamic Classifier Chains

Algorithm for Double-Layer Structure Multi-label Classification with Optimal Sequence Based on Attention Mechanism

Contact Info

Product

Resources

About