Tackling Virtual and Real Concept Drifts: An Adaptive Gaussian Mixture Model Approach

Abstract

Real-world applications have been dealing with large amounts of data that arrive over time and generally present changes in their underlying joint probability distribution, i.e., concept drift. Concept drift can be subdivided into two types: virtual drift, which affects the unconditional probability distribution p(x), and real drift, which affects the conditional probability distribution p(y|x). Existing works focuses on real drift. However, strategies to cope with real drift may not be the best suited for dealing with virtual drift, since the real class boundaries remain unchanged. We provide the first in depth analysis of the differences between the impact of virtual and real drifts on classifiers' suitability. We propose an approach to handle both drifts called On-line Gaussian Mixture Model With Noise Filter For Handling Virtual and Real Concept Drifts (OGMMF-VRD). Experiments with seven synthetics and seven real-world datasets show that OGMMF-VRD outperforms other approaches with separate mechanisms to deal with virtual and real drifts. It also has more stable rankings and smaller drops in performance during drifting periods than existing ensemble approaches, thus being more reliable for adoption in practice.

Dynamic Disagreeing Neighbors: A complexity measure for instance selection

“…Using the information provided by the kDN and the error of each instance, they present two strategies to select minority samples: assuming that the most complex instances offer more information or assuming that the easiest instances contain more information. In [23], they present an approach to deal with both real and virtual concept drifts where the kDN is used as a noise filter in two different steps. They consider that instances with the kDN greater than 0.8 (using k = 5) should not be considered since the 80% of their neighbors are not from their same class and, therefore, are noisy instances that can potentially harm the system.…”

“…The kDN of an instance is defined as the percentage of its k nearest neighbors that belong to other classes [29]. For example, it has been successfully applied as a noise filter in an online scheme [23]. In the same way, in [31], the authors employ the kDN by filtering the instances with a high value since they can be safely discarded because they are either noisy instances or outliers.…”

Online Boosting Adaptive Learning under Concept Drift for Multistream Classification

“…To achieve this, we employ the Drift Detection Method (DDM) (Gama et al 2004) for labeled source streams, as it offers a stable and accurate detection approach. Simultaneously, we utilize the Gaussian Mixture Model (GMM) (Oliveira, Minku, and Oliveira 2021) based weighting strategy for asynchronous drift adaptation in these streams. For the unlabeled target stream, we design two sliding windows and continuously monitor their distribution changes to effectively detect drift occurrences.…”

A Novel Data Stream Learning Approach to Tackle One-Sided Label Noise From Verification Latency

“…Frenay et al [10] extended a prototype-based online classifier in a probabilistic manner to deal with label noise. Oliveira et al [11] adopted a filter based on data hardness techniques, but requires a window of data to be stored for this purpose. There are also studies investigating "label" noise in regression problems under online learning scenarios without verification latency [12], [13].…”