A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework

Abstract

Class imbalance poses new challenges when it comes to classifying data streams. Many algorithms recently proposed in the literature tackle this problem using a variety of data-level, algorithm-level, and ensemble approaches. However, there is a lack of standardized and agreed-upon procedures on how to evaluate these algorithms. This work presents a taxonomy of algorithms for imbalanced data streams and proposes a standardized, exhaustive, and informative experimental testbed to evaluate algorithms in a collection of diverse and challenging imbalanced data stream scenarios. The experimental study evaluates 24 state-of-the-art data streams algorithms on 515 imbalanced data streams that combine static and dynamic class imbalance ratios, instance-level difficulties, concept drift, real-world and semi-synthetic datasets in binary and multi-class scenarios. This leads to the largest experimental study conducted so far in the data stream mining domain. We discuss the advantages and disadvantages of state-of-the-art classifiers in each of these scenarios and we provide general recommendations to end-users for selecting the best algorithms for imbalanced data streams. Additionally, we formulate open challenges and future directions for this domain. Our experimental testbed is fully reproducible and easy to extend with new methods. This way we propose first standardized approach to conducting experiments in imbalanced data streams that can be used by other researchers to create trustworthy and fair evaluation of newly proposed methods. Our experimental framework can be downloaded from https://github.com/canoalberto/imbalanced-streams.

Nonstationary data stream classification with online active learning and siamese neural networks✩

“…The combined problem of class imbalance and concept drift remains an open challenge [21,3]. There are two types of approaches, algorithm-level and datalevel.…”

“…Cost-sensitive learning, one-class classification, and anomaly detection are other types of methods that can handle imbalance. We direct the interested reader towards these excellent surveys [21,3].…”

Toward Privacy Preservation Using Clustering Based Anonymization: Recent Advances and Future Research Outlook

“…The proposed method can distinguish between normal and anomalous images with 98.52% accuracy. Recently, training/learning high-quality AI models from imbalanced data in real-time applications has become a popular research topic [333]. Vu et al [334] developed a novel collaborative data model for semi-fully distributed settings for real-time medical applications.…”

The Influence of Multiple Classes on Learning Online Classifiers from Imbalanced and Concept Drifting Data Streams

“…Also all experiments were conducted in MOA command line mode similarly to ones described in [4]. 2 Recall that our study aims at detailed investigating the impact of selected factors on the classification of multi-class imbalanced stream, not at comparing many different classifiers such as [1], so it is sufficient to select few representative classifiers only.…”

“…However in evolving data streams they could also influence the changes in local class distributions and other local drifts. Nevertheless, the conjunctions of these data factors and drifts have not been sufficiently studied yet, see discussions in [1,6,4].…”