Online sequential extreme studentized deviate tests for anomaly detection in streaming data with varying patterns

Ryu, Minho; Lee, Geonseok; Lee, Kichun

doi:10.1007/s10586-021-03236-0

Cited by 5 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, n outliers [39]. Thus, the GESD performs better in outlier detection in univariate data sets with varying patterns [40]. Performing the time-series analysis per extraction point has several upsides.…”

Section: Extraction and Processing Of Velocity Data For Time Series A...mentioning

confidence: 99%

Automated Detection of Glacier Surges from Sentinel-1 Surface Velocity Time Series—An Example from Svalbard

et al. 2023

View full text Add to dashboard Cite

Even though surge-type glaciers make up only a small percentage of all glaciers, related research contributes considerably to the general understanding of glacier flow mechanisms. Recent studies based on remote sensing techniques aimed to disentangle underlying processes related to glacier surges. They have proven the possibilities yielded by combining high performance computing and earth observation. In addition, modelling approaches to surges have seen increasing popularity, yet large spatial and temporal data about timing of surge incites are missing. We aimed to develop an algorithm that not only detects surge type glaciers but also determines the timing of a surge onset, while being computationally inexpensive, transferable, and expandable in time and space. The algorithm is based on time series analyses of glacier surface velocity derived from Sentinel-1 data. After seasonal and trend decomposition, outlier detection is performed by the General Studentized Extreme Deviate Test, an iterative algorithm well suited for outlier detection in univariate time series. To determine surges, cluster analysis is performed to identify outlier clusters, which are linked to glacier surges. We demonstrate the viability on the Svalbard archipelago for the period 2015 to 2021 where we have identified 18 glacier surges and the timing of their active phase.

show abstract

Section: Extraction and Processing Of Velocity Data For Time Series A...mentioning

confidence: 99%

Automated Detection of Glacier Surges from Sentinel-1 Surface Velocity Time Series—An Example from Svalbard

et al. 2023

View full text Add to dashboard Cite

show abstract

“…When the transaction with label feature, the results returned by the above algorithms may lack signifcance and some patterns are not statistically meaningful. SSPM algorithms look for signifcant patterns and have been widely used in e-commerce searching [26], essential protein recognition [27], and community detection [28,29]. Hämäläinen [30] proposes the SSPM model frst and regards signifcant pattern mining as a multiple-hypothesis testing problem.…”

Section: Related Workmentioning

confidence: 99%

Discovering Significant Sequential Patterns in Data Stream by an Efficient Two-Phase Procedure

Tang

Wang

Liu

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

One essential topic of mining sequential patterns in the data stream is to optimize the time-space computations. However, more importantly, it should pay more attention to the significance of mining results as a large portion of them just response to the user-defined constraints purely by accident and they may have no statistical significance. In this paper, we propose FSSPDS, an efficient two-phase algorithm to discover the significant sequential patterns (SSPs) in the data stream with typical sliding windows, which has never been considered in existing problems. First, for generating SSPs candidates with high-quality, FSSPDS takes testable support and pattern length constraints into account and insignificant patterns were removed timely by a pattern-growth method. In the second phase, appropriate permutation testing is used to test the significance of the SSPs candidates. Exact permutation p values are obtained in a novel combination way based on unconditional Barnard’s test statistic which better reflects the process of data generations and collections. Experimental evaluations show that FSSPDS allows the discovery of SSPs in the data stream and rivals the state-of-the-art approaches efficiently under the control of family-wise error rate (FWER), especially for time efficiency, which was approximately an order of magnitude higher.

show abstract

“…Proposed IoTSDA method uses the STL algorithm by using the Loess [9][10] and a piecewise median approach leveraged by the Twitter Inc [11] [12] [13]. Interquartile range (IQR) [14] [15] and generalized extreme studentized deviate (GESD) [16] are used with IoTSDA to create rank-based bands for segregating anomalies form the time series data set.…”

Section: Introductionmentioning

confidence: 99%

IoTSDA: IoT-Edge analytics architecture for anomaly detection of univariate health data

2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Presence of abnormal data points can create noise in a time series data set. Such an issue can be exaggerated in the internet of things (IoT)-based scenario that collects sensor data in a regular interval. Traditional machine and deep learning algorithms require complex memory footprint and computational overhead, thus resulting in the nonconfirmative design aspects of the IoT-based ecosystem. This article aims at detection of anomalies from a time series health data set collected from a pulse sensor integrated IoT-based device. We use the seasonal-decomposition by Loess i.e. STL and piecewise median schemes along with the proposed IoTSDA method. We implement "Anomalize " package on top of an "IRKernel" enabled R platform on the Raspberry Pi 4 that acts as the IoT-edge device. The deployed IoTSDA method supports almost instantaneous anomaly detection with help of interquartile range and generalized extreme studentized deviate statistics. Results show promising behavior of anomaly detection from the implied data set containing pulse analog raw values. The proposed IoTSDA method leverages an IoT-edge integrated analytics platform for smart health care decision making models.

show abstract

Online sequential extreme studentized deviate tests for anomaly detection in streaming data with varying patterns

Cited by 5 publications

References 17 publications

Automated Detection of Glacier Surges from Sentinel-1 Surface Velocity Time Series—An Example from Svalbard

Automated Detection of Glacier Surges from Sentinel-1 Surface Velocity Time Series—An Example from Svalbard

Discovering Significant Sequential Patterns in Data Stream by an Efficient Two-Phase Procedure

IoTSDA: IoT-Edge analytics architecture for anomaly detection of univariate health data

Contact Info

Product

Resources

About