Multi-label classification via multi-target regression on data streams

Abstract

Multi-label classification (MLC) tasks are encountered more and more frequently in machine learning applications. While MLC methods exist for the classical batch setting, only a few methods are available for streaming setting. In this paper, we propose a new methodology for MLC via multi-target regression in a streaming setting. Moreover, we develop a streaming multi-target regressor iSOUP-Tree that uses this approach. We experimentally compare two variants of the iSOUP-Tree method (building regression and model trees), as well as ensembles of iSOUP-Trees with state-of-the-art tree and ensemble methods for MLC on data streams. We evaluate these methods on a variety of measures of predictive performance (appropriate for the MLC task). The ensembles of iSOUP-Trees perform significantly better on some of these measures, especially the ones based on label ranking, and are not significantly worse than the competitors on any of the remaining measures. We identify the thresholding problem for the task of MLC on data streams as a key issue that needs to be addressed in order to obtain even better results in terms of predictive performance.

A Novel Online Stacked Ensemble for Multi-Label Stream Classification

“…The evaluation of ensembles are started after the first learner in the ensemble is formed. We used fixed number of 10 classifiers as the ensemble size, mimicking the previously conducted experiments to enable comparison [19,22]. For incremental evaluation of the classifiers, we used window-based evaluation with the window size {100, 250, 500, 1000} according to the size of datasets 2 .…”

“…We have 7 baseline models. Four of the baselines use fixed-sized windows with no concept drift detecting mechanism: EBR [24], ECC [24], EPS [23], EBRT [19], whereas 3 of them use ADWIN as their concept drift detector: EaBR [22], EaCC [22], and EaPS [22]). In all models, BR and CC transformations use a Hoeffding Tree classifier whereas the PS transformation uses a Naive Bayes classifier.…”

“…Modifying the split criterion of Hoeffding Trees for multi-label entropy, Multi-label Hoeffding Trees [22] are developed. More recently, a novel decision tree based method, iSOUP-Trees (incremental Structured Output Prediction Tree) [19] are proposed where adaptive perceptrons are placed in the leaves of incremental trees and the perceptrons' weights are used in producing a prediction that is a linear combination of the input's attributes. In a nutshell, "in Problem Transformation, data is modified to make it suitable for algorithms; whereas in Algorithm Adaptation, algorithms are modified to make them suitable for data" [37].…”

“…In fact, the phrase 'Ensemble of' in the field usually means that it is the OzaBagged version of the base classifier that is mentioned. EBR [24], ECC [24], EPS [23] and EBRT [19] (Ensembles of BR, CC, PS and iSOUP Regression Trees respectively) are examples of this convention.…”

“…Our experiments are conducted on 7 datasets 1 that are from diverse application domains (genes, newspapers, aviation safety reports and so on), given in Table 3. These datasets are extensively used in the literature [19,22,27].…”