A Shape-Based Method for Concept Drift Detection and Signal Denoising

Hinder, Fabian; Brinkrolf, Johannes; Vaquet, Valerie; Hammer, Barbara

doi:10.1109/ssci50451.2021.9660111

Cited by 4 publications

(19 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notice, that this is the specific instantiation of the original definition by [27], which focuses on consecutive time windows. Assuming the samples arrive with equidistant delay, we may estimate σ based on the last 2n l samples, where n l is the number of samples in a time period of length l. In [27] d is chosen as the total variation norm, in [6] Hellinger distance is used, [25] and [16] use the Maximum Mean Discrepancy (MMD) [13], [5] use Kullback-Leibler divergence, [24] use the (feature wise) maximum-norm of the cdfs, and [12] use classification accuracy of a linear model, which is thus related to the total variance and MMD (with linear kernel).…”

Section: Problem Setup and Related Workmentioning

confidence: 99%

“…In [16] the theoretical properties of σ are analyzed. The main results are that σ is indeed able to identify drift for a valid distance function by taking on values larger than zero, which also points out the point in time.…”

Section: Problem Setup and Related Workmentioning

confidence: 99%

“…However, those effects are usually spread out in time. Although the work [16] identifies a characteristic shape of this function in case of isolated, abrupt drift events, a theoretical description of the shape in general is still an open question.…”

Section: Problem Setup and Related Workmentioning

confidence: 99%

“…Thus, many algorithms consider squared, overlaying, weighted functions that are derived from the drift process. Therefore, it is reasonable to assume that intricate patterns that occur in drift induced signals are a result of the used weighting; indeed, in [16] it is shown that abrupt drift events are distributed in time by the drift magnitude. In the following we will make this more explicit by restricting ourselves to the case of a convectional weighting scheme and consider drift in the formal framework of (generalized) derivatives.…”

Section: The Shape Of the Drift Magnitudementioning

confidence: 99%

“…Many unsupervised drift detection schemes rely on a comparison of distributions taken from two consecutive time windows [11]. These techniques can easily lead to false positives; moreover, they are not capable of dealing with consecutive drift events in close proximity [16].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Precise Change Point Detection using Spectral Drift Detection

Hinder¹,

Artelt²,

Vaquet³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The notion of concept drift refers to the phenomenon that the data generating distribution changes over time; as a consequence machine learning models may become inaccurate and need adjustment. In this paper we consider the problem of detecting those change points in unsupervised learning. Many unsupervised approaches rely on the discrepancy between the sample distributions of two time windows. This procedure is noisy for small windows, hence prone to induce false positives and not able to deal with more than one drift event in a window. In this paper we rely on structural properties of drift induced signals, which use spectral properties of kernel embedding of distributions. Based thereon we derive a new unsupervised drift detection algorithm, investigate its mathematical properties, and demonstrate its usefulness in several experiments.

show abstract

Section: Problem Setup and Related Workmentioning

confidence: 99%

Section: Problem Setup and Related Workmentioning

confidence: 99%

Section: Problem Setup and Related Workmentioning

confidence: 99%

Section: The Shape Of the Drift Magnitudementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Precise Change Point Detection using Spectral Drift Detection

Hinder¹,

Artelt²,

Vaquet³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

System for Continuous Multi-Dimensional Mobile Network KPI Tracking and Prediction in Drifting Environments

Schippers

Böcker

Wietfeld

2023

2023 IEEE International Systems Conference (SysCon)

View full text Add to dashboard Cite

One or two things we know about concept drift—a survey on monitoring in evolving environments. Part A: detecting concept drift

Hinder,

Vaquet,

Hammer

2024

Front. Artif. Intell.

View full text Add to dashboard Cite

The world surrounding us is subject to constant change. These changes, frequently described as concept drift, influence many industrial and technical processes. As they can lead to malfunctions and other anomalous behavior, which may be safety-critical in many scenarios, detecting and analyzing concept drift is crucial. In this study, we provide a literature review focusing on concept drift in unsupervised data streams. While many surveys focus on supervised data streams, so far, there is no work reviewing the unsupervised setting. However, this setting is of particular relevance for monitoring and anomaly detection which are directly applicable to many tasks and challenges in engineering. This survey provides a taxonomy of existing work on unsupervised drift detection. In addition to providing a comprehensive literature review, it offers precise mathematical definitions of the considered problems and contains standardized experiments on parametric artificial datasets allowing for a direct comparison of different detection strategies. Thus, the suitability of different schemes can be analyzed systematically, and guidelines for their usage in real-world scenarios can be provided.

show abstract

A Shape-Based Method for Concept Drift Detection and Signal Denoising

Cited by 4 publications

References 14 publications

Precise Change Point Detection using Spectral Drift Detection

Precise Change Point Detection using Spectral Drift Detection

System for Continuous Multi-Dimensional Mobile Network KPI Tracking and Prediction in Drifting Environments

One or two things we know about concept drift—a survey on monitoring in evolving environments. Part A: detecting concept drift

Contact Info

Product

Resources

About