An Adaptive Outlier Detection Technique for Data Streams

Sadik, Shiblee; Gruenwald, Le

doi:10.1007/978-3-642-22351-8_52

Cited by 4 publications

(3 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Detection and removal of outliers are necessary to improve the performance of machine learning algorithms ( Ester et al, 1996 ; Syafrudin et al, 2018 ). We adopted a method based on the interquartile range ( Acuna and Rodriguez, 2004 ; Sadik and Gruenwald, 2011 ; Yin et al, 2016 ). An accelerometer data point was considered an outlier if it was outside of the upper/lower fence and beyond the accelerometer range (=16 × 9.81 m/s 2 ).…”

Section: Methodsmentioning

confidence: 99%

Improved pig behavior analysis by optimizing window sizes for individual behaviors on acceleration and angular velocity data

Alghamdi

Zhao

et al. 2022

Journal of Animal Science

View full text Add to dashboard Cite

This paper presents the application of machine learning algorithms to identify pigs’ behaviors from data collected using the wireless sensor nodes mounted on pigs. The sensor node attached to a pig’s back senses the acceleration and angular velocity in three axes, and the sensed data is transmitted to a host computer wirelessly. Two video cameras, one attached to the ceiling of the pigpen and the other one to a fence, provided ground truth for data annotations. The data was collected from pigs for 131 hours over two months. As the typical behavior period depends on the behavior type, we segmented the acceleration data with different window sizes (WS) and step sizes (SS), and tested how the classification performance of different activities varied with different WS and SS. After exploring the possible combinations, we selected the optimum WS and SS. To compare performance, we used five machine learning algorithms, specifically support vector machine, k-nearest neighbors, decision tree, naïve Bayes, and random forest. Among the five algorithms, random forest achieved the highest F1 score for four major behaviors consisting of 92.36% in total. The F1 scores of the algorithm were 0.98 for ‘eating’, 0.99 for ‘lying’, 0.93 for ‘walking’, and 0.91 for ‘standing’ behaviors. The optimal WS was 7 seconds for ‘eating’ and ‘lying’, and 3 seconds for ‘walking’ and ‘standing’. The proposed work demonstrates that, based on the length of behavior, the adaptive window and step sizes increase the classification performance.

show abstract

Section: Methodsmentioning

confidence: 99%

Improved pig behavior analysis by optimizing window sizes for individual behaviors on acceleration and angular velocity data

Alghamdi

Zhao

et al. 2022

Journal of Animal Science

View full text Add to dashboard Cite

show abstract

“…The analysis parameter is the total cell volume. The comparison algorithms are the automatic outlier detection for data streams (A-ODDS) [30,31] and SNWCAD-DS algorithms. The adopted standard is the area under the curve (AUC) and the Jaccard similarity coefficient.…”

Section: Simulation and Comparisonmentioning

confidence: 99%

The Application of a Double CUSUM Algorithm in Industrial Data Stream Anomaly Detection

Wang

Yue

2018

Symmetry

View full text Add to dashboard Cite

The effect of the application of machine learning on data streams is influenced by concept drift, drift deviation, and noise interference. This paper proposes a data stream anomaly detection algorithm combined with control chart and sliding window methods. This algorithm is named DCUSUM-DS (Double CUSUM Based on Data Stream), because it uses a dual mean value cumulative sum. The DCUSUM-DS algorithm based on nested sliding windows is proposed to satisfy the concept drift problem; it calculates the average value of the data within the window twice, extracts new features, and then calculates accumulated and controlled graphs to avoid misleading by interference points. The new algorithm is simulated using drilling engineering industrial data. Compared with automatic outlier detection for data streams (A-ODDS) and with sliding nest window chart anomaly detection based on data streams (SNWCAD-DS), the DCUSUM-DS can account for concept drift and shield a small amount of interference deviating from the overall data. Although the algorithm complexity increased from 0.1 second to 0.19 second, the classification accuracy receiver operating characteristic (ROC) increased from 0.89 to 0.95. This meets the needs of the oil drilling industry data stream with a sampling frequency of 1 Hz, and it improves the classification accuracy.

show abstract

“…Therefore, the online classification accuracy should be improved as much as possible, which is the principle of data stream anomaly detection. The proposed algorithm is compared with DBOD-DS [35] and A-ODDS [36]. The AUC (Area under Curve) index and the Jaccard coefficient in the receiver operating characteristic curve (ROC) are employed as a performance indicator.…”

Section: Simulation and Comparisonmentioning

confidence: 99%

Application of Sliding Nest Window Control Chart in Data Stream Anomaly Detection

Wang

Liang

et al. 2018

Symmetry

View full text Add to dashboard Cite

Since data stream anomaly detection algorithms based on sliding windows are sensitive to the abnormal deviation of individual interference data, this paper presents a sliding nest window chart anomaly detection based on the data stream (SNWCAD-DS) by employing the concept of the sliding window and control chart. By nesting a small sliding window in a large sliding window and analyzing the deviation distance between the small window and the large sliding window, the algorithm increases the out-of-bounds detection ratio and classifies the conceptual drift data stream online. The designed algorithm is simulated on the industrial data stream of drilling engineering. The proposed algorithm SNWCAD is compared with Automatic Outlier Detection for Data Streams (A-ODDS) and Distance-Based Outline Detection for Data Stream (DBOD-DS). The experimental results show that the new algorithm can obtain higher detection accuracy than the compared algorithms. Furthermore, it can shield the influence of individual interference data and satisfy actual engineering needs.

show abstract

An Adaptive Outlier Detection Technique for Data Streams

Cited by 4 publications

References 3 publications

Improved pig behavior analysis by optimizing window sizes for individual behaviors on acceleration and angular velocity data

Improved pig behavior analysis by optimizing window sizes for individual behaviors on acceleration and angular velocity data

The Application of a Double CUSUM Algorithm in Industrial Data Stream Anomaly Detection

Application of Sliding Nest Window Control Chart in Data Stream Anomaly Detection

Contact Info

Product

Resources

About