Combining block-based and online methods in learning ensembles from concept drifting data streams

Brzeziński, Dariusz; Stefanowski, Jerzy

doi:10.1016/j.ins.2013.12.011

Cited by 159 publications

(84 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3 For the real-world datasets we chose four data streams which are commonly used as benchmarks [8,25,34,43]. More precisely, we chose Airlines (Air) and Electricity (Elec) as examples of fairly balanced datasets, and KDDCup and PAKDD as examples of moderately imbalanced datasets.…”

Section: Datasetsmentioning

confidence: 99%

Prequential AUC: properties of the area under the ROC curve for data streams with concept drift

2017

Self Cite

View full text Add to dashboard Cite

Modern data-driven systems often require classifiers capable of dealing with streaming imbalanced data and concept changes. The assessment of learning algorithms in such scenarios is still a challenge, as existing online evaluation measures focus on efficiency, but are susceptible to class ratio changes over time. In case of static data, the area under the receiver operating characteristics curve, or simply AUC, is a popular measure for evaluating classifiers both on balanced and imbalanced class distributions. However, the characteristics of AUC calculated on time-changing data streams have not been studied. This paper analyzes the properties of our recent proposal, an incremental algorithm that uses a sorted tree structure with a sliding window to compute AUC with forgetting. The resulting evaluation measure, called prequential AUC, is studied in terms of: visualization over time, processing speed, differences compared to AUC calculated on blocks of examples, and consistency with AUC calculated traditionally. Simulation results show that the proposed measure is statistically consistent with AUC computed traditionally on streams without drift and comparably fast to existing evaluation procedures. Finally, experiments on real-world and synthetic data showcase characteristic properties of prequential AUC compared to classification accuracy, G-mean, Kappa, Kappa M, and recall when used to evaluate classifiers on imbalanced streams with various difficulty factors.

show abstract

Section: Datasetsmentioning

confidence: 99%

Prequential AUC: properties of the area under the ROC curve for data streams with concept drift

2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…The Online Accuracy Updated Ensemble (OAUE) [8] maintains a weighted set of component classifiers, such that the weighting is given by an adaptation to incremental learning of the weight function presented in [25]. Every d instances, the least accurate classifier is replaced by a candidate classifier, which has been trained only in the last d instances.…”

Section: Ensemble Methods For Data Stream Classificationmentioning

confidence: 99%

“…Ensemble classifiers achieve high accuracy through the combination of a diverse set of component classifiers, such that (ideally) incorrect predictions are obfuscated while correct predictions are highlighted. In a data stream scenario, ensemble classifiers also have the advantageous characteristic of being flexible, i.e., it is possible to replace (or remove) component classifiers based on drift detector algorithms [5,6] or other methods [8,12,13,18]. Even though there is not a solid proof of a strong correlation between accuracy and diversity [19,20], it is possible to verify that a set of homogeneous classifiers that always decide for the same labels cannot achieve better (or worse) results than one of them alone.…”

Section: Introductionmentioning

confidence: 99%

“…It is important to note that diversity is favorable as long as the individual decisions are properly combined. Existing ensemble classifiers for data streams use various techniques for combining predictions, such as majority vote, weighted majority [21] and other methods [2,8,13] that attempt to incorporate characteristics, mainly to account for concept drifts, that benefit classifiers adapted to the current concept over others.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pairwise combination of classifiers for ensemble learning on data streams

Gomes

Barddal

Enembreck

2015

Proceedings of the 30th Annual ACM Symposium on Applied Computing

View full text Add to dashboard Cite

This work presents two different voting strategies for ensemble learning on data streams based on pairwise combination of component classifiers. Despite efforts to build a diverse ensemble, there is always some degree of overlap between component classifiers models. Our voting strategies are aimed at using these overlaps to support ensemble prediction. We hypothesize that by combining pairs of classifiers it is possible to alleviate incorrect individual predictions that would otherwise negatively impact the overall ensemble decision. The first strategy, Pairwise Accuracy (PA), combines the shared accuracy estimation of all possible pairs in the ensemble, while the second strategy, Pairwise Patterns (PP), record patterns of pairwise decisions during training and use these patterns during prediction. We present empirical results comparing ensemble classifiers with their original voting methods and our proposed methods in both real and synthetic datasets, with and without concept drifts. Our analysis indicates that pairwise voting is able to enhance overall performance for PP, especially on real datasets, and that PA is useful whenever there are noticeable differences in accuracy estimates among ensemble members, which is common during concept drifts.

show abstract

“…Kolter and Maloof proposed the dynamic weighted majority algorithm [15] and the additive expert ensemble algorithm [16] for online learning in succession. Brzezinski and Stefanowski [17] proposed the online accuracy updated ensemble (OAUE) algorithm, which uses the proposed function to incrementally train and weight component classifiers.…”

Section: Related Workmentioning

confidence: 99%

A Tree-based Concept Drift Detection Method by Three-way Decisions

Su¹

2017

Proceedings of the 2017 2nd International Conference on Automation, Mechanical Control and Computational Engineering (AMCCE 201

View full text Add to dashboard Cite

Abstract. Concept drift detection is an active research area in data stream mining. The existing concept drift detection methods usually determine a concept is drifted or not drifted. In other words, the concepts are assigned into two classes such as drifted and un-drifted, which is typically based on two-way decisions. Most likely, these methods incorrectly determine that concept drift occurs due to some uncertain factors such as noise, and some real concept drifts are not detected. Inspired by the three-way decision theory, we propose a tree-based concept drift detection method by three-way decisions in this paper, in order to improve the accuracy of detection. The basic idea of the method is to assign the concepts into three classes such as drifted, nondeterministic drifted and un-drifted. Furthermore, concepts in the class of nondeterministic drifted are determined further according to the deviation between the classification error rates. The results of comparison experiments show the effectiveness and efficiency of the proposed methods.

show abstract

Combining block-based and online methods in learning ensembles from concept drifting data streams

Cited by 159 publications

References 16 publications

Prequential AUC: properties of the area under the ROC curve for data streams with concept drift

Prequential AUC: properties of the area under the ROC curve for data streams with concept drift

Pairwise combination of classifiers for ensemble learning on data streams

A Tree-based Concept Drift Detection Method by Three-way Decisions

Contact Info

Product

Resources

About