Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains

Abstract

Classifier chains is a key technique in multi-label classification, since it allows to consider label dependencies effectively. However, the classifiers are aligned according to a static order of the labels. In the concept of dynamic classifier chains (DCC) the label ordering is chosen for each prediction dynamically depending on the respective instance at hand. We combine this concept with the boosting of extreme gradient boosted trees (XGBoost), an effective and scalable state-of-the-art technique, and incorporate DCC in a fast multi-label extension of XGBoost which we make publicly available. As only positive labels have to be predicted and these are usually only few, the training costs can be further substantially reduced. Moreover, as experiments on eleven datasets show, the length of the chain allows for more control over the usage of previous predictions and hence over the measure one wants to optimize.

AdaBoost.C2: Boosting Classifiers Chains for Multi-Label Classification

“…Notes: Base for base classifier, Region for application region, 1 for(Freund and Schapire 1997), 2 for(Johnson and Cipolla 2005), 3 for (Al-Salemi, Noah, and Ab Aziz 2016), 4 for(Wang et al 2019), 5 for(Si et al 2017), 6 for(Cheng et al 2020), 7 for(Bohlender, Loza Mencía, and Kulessa 2020), 8 for (Zhang and Jung 2020), 9 for(Amit, Dekel, and Singer 2007), 10 for(Rapp et al 2021), 11 for (Dembczynski, Kotlowski, and Hüllermeier 2012), 12 for(Jung and Tewari 2018).…”