WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data

Faber, Kamil; Corizzo, Roberto; Śnieżyński, Bartłomiej; Baron, Michael; Japkowicz, Nathalie

doi:10.1109/bigdata52589.2021.9671962

Cited by 15 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Components of the SG-SRI system (Sur et al, 2022;Daniels et al, 2022) include: (i) WATCH (Faber et al, 2021(Faber et al, , 2022, a Wasserstein-based statistical changepoint detection that detects changes in the environment; (ii) Self-Taught Associative Memory (STAM) (Smith et al, 2021), to generate feature maps from RGB images in a continually updated manner; (iii) Danger detection, using the continual learner deep streaming linear discriminant analysis (DeepSLDA) ; (iv) Compression, using the REMIND algorithm that uses Product Quantization (PQ); and (v) Sleep phase, implemented using the Eigentask framework (Raghavan et al, 2020).…”

Section: System Group Sri -Starcraft II 551 System Overviewmentioning

confidence: 99%

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Baker¹,

New²,

Aguilar-Simon³

et al. 2023

Preprint

View full text Add to dashboard Cite

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development -both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

show abstract

Section: System Group Sri -Starcraft II 551 System Overviewmentioning

confidence: 99%

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Baker¹,

New²,

Aguilar-Simon³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…denotes the Euclidean distance and (X, Y ) is a pair of random variables having distribution f ∈ Γ . The implementation of this distance has technical issues [10], and the usual approach is to get approximations of the theoretical value. This is provided by available packages, like the one used in this paper (see below).…”

Section: Re-visiting the Concepts On Dissimilaritiesmentioning

confidence: 99%

“…(ii) Second, we empirically analyze the performance of our drift detector using three different and important dissimilarity metrics: KLdivergence, Hellinger distance and Wasserstein distance. The selection of these metrics is based on the fact that nowadays are often used in the learning area [5,9,10].…”

Section: Introductionmentioning

confidence: 99%

Experimental Analysis on Dissimilarity Metrics and Sudden Concept Drift Detection

Basterrech

Platoš

Rubino

et al. 2023

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Learning from non-stationary data presents several new challenges. Among them, a significant problem comes from the sudden changes in the incoming data distributions, the so-called concept drift. Several concept drift detection methods exist, generally based on distances between distributions, either arbitrarily selected or context-dependent. This paper presents a straightforward approach for detecting concept drift based on a weighted dissimilarity metric over posterior probabilities. We also evaluate the performance of three well-known dissimilarity metrics when used by the proposed approach. Experimental evaluation has been done over ten datasets with injected sudden drifts in a binary classification context. Our results first suggest choosing the Kullback-Leibler divergence, and second, they show that our drift detection procedure based on dissimilarity measures is pretty efficient.

show abstract

“…This does not readily scale to a large number of dimensions (26; 33; 32). Indeed, many change detection algorithms tend to perform well on univariate data and on low-dimensional multivariate data (20), but their detection quality drops as the number of dimensions increases (11).…”

Section: Related Workmentioning

confidence: 99%

“…In our experiments, we compare MMDAW to ADWINK, IBDD (7), WATCH (11), and D3 (16). They all explicitly target at change detection in high-dimensional data.…”

Section: Related Workmentioning

confidence: 99%

Scalable Online Change Detection for High-dimensional Data Streams

Kalinke¹,

Heyden²,

Fouché³

et al. 2022

Preprint

View full text Add to dashboard Cite

Detecting changes in data streams is a core objective in their analysis and has applications in, say, predictive maintenance, fraud detection, and medicine. A principled approach to detect changes is to compare distributions observed within the stream to each other. However, data streams often are high-dimensional, and changes can be complex, e.g., only manifest themselves in higher moments. The streaming setting also imposes heavy memory and computation restrictions. We propose an algorithm, Maximum Mean Discrepancy Adaptive Windowing (MMDAW), which leverages the well-known Maximum Mean Discrepancy (MMD) two-sample test (18), and facilitates its efficient online computation on windows whose size it flexibly adapts. As MMD is sensitive to any change in the underlying distribution, our algorithm is a general-purpose non-parametric change detector that fulfills the requirements imposed by the streaming setting. Our experiments show that MMDAW achieves better detection quality than state-of-the-art competitors.Preprint. Under review.

show abstract

WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data

Cited by 15 publications

References 39 publications

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Experimental Analysis on Dissimilarity Metrics and Sudden Concept Drift Detection

Scalable Online Change Detection for High-dimensional Data Streams

Contact Info

Product

Resources

About