Anomaly detection in streaming environmental sensor data: A data-driven modeling approach

Hill, David; Minsker, Barbara

doi:10.1016/j.envsoft.2009.08.010

Cited by 271 publications

(133 citation statements)

References 31 publications

Supporting

Mentioning

131

Contrasting

Unclassified

Order By: Relevance

“…The clustering based method depends on the number of clusters and the existence of outliers in the data. The authors of Reference [32] think that outlier detection based on time series forecasting is the most simple and intuitive method, but the predictive ability of this method depends on the prediction model, and it is difficult to determine a reasonable threshold.…”

Section: Outlier Detectionmentioning

confidence: 99%

Correction of Outliers in Temperature Time Series Based on Sliding Window Prediction in Meteorological Sensor Network

Wang

2017

Information

View full text Add to dashboard Cite

Abstract:In order to detect outliers in temperature time series data for improving data quality and decision-making quality related to design and operation, we proposed an algorithm based on sliding window prediction. Firstly, the time series are segmented based on the sliding window. Then, the prediction model is established based on the history data to predict the future value. If the difference between a predicted value and a measured value is larger than the preset threshold value, the sequence point will be judged to be an outlier and then corrected. In this paper, the sliding window and parameter settings of the algorithm are discussed and the algorithm is verified on actual data. This method does not need to pre classify the abnormal points and perform fast, and can handle large scale data. The experimental results show that the proposed algorithm can not only effectively detect outliers in the time series of meteorological data but also improves the correction efficiency notoriously.

show abstract

Section: Outlier Detectionmentioning

confidence: 99%

Correction of Outliers in Temperature Time Series Based on Sliding Window Prediction in Meteorological Sensor Network

Wang

2017

Information

View full text Add to dashboard Cite

show abstract

“…Some examples of elementary ones are ARIMA model [9] and linear neural network. There are also some works comparing use of multilayer perceptron with linear neural network, naïve predictor and nearest cluster [10]. Other solution can be Gibbs probes [11] or weighted maximal likelihood estimation [12].…”

Section: Available Algorithmsmentioning

confidence: 99%

Anomaly detection in server metrics with use of one-sided median algorithm

Zacher¹,

Ryba

2017

Journal of Applied Computer Science Methods

View full text Add to dashboard Cite

In this paper we consider the problem of anomaly detection over time series metrics data took from one of corporate grade mail service cluster. We propose the algorithm based on one-sided median concept and present some results of experiments showing impact of parameters settings on algorithm performance. In addition we present short description of classes of anomalies discovered in monitored system. Proposed one-sided median based algorithm shows great robustness and good detection rate and can be considered as possible simple production ready solution.

show abstract

“…If it is not, the value is considered anomalous. This algorithm is based on an approach described by Hill and Minsker [3]. It effectively detects changes in both average value and variance, but rather than relying on a continuous window of allowable data it uses the cluster centroids to define regions of acceptable metasenor values.…”

Section: Algorithmsmentioning

confidence: 99%

Automatic real-time fault detection for industrial assets using metasensors

Butters¹,

Güttel

Shapiro³

et al. 2015

Asset Management Conference 2015

View full text Add to dashboard Cite

Large-scale industrial plants require physical sensors to continuously measure quantities such as temperatures or pressures. A large number of sensors is required to accurately describe the operating state of the plant, which unfortunately makes it very difficult for them to be effectively monitored by human operators. In this work we present a method to construct so-called metasensors, virtual sensors that compress the information from several sensors in an optimal manner. These metasensors are used as inputs to a novel anomaly detection system that automatically alerts operators to abnormal operation behaviour.

show abstract

Anomaly detection in streaming environmental sensor data: A data-driven modeling approach

Cited by 271 publications

References 31 publications

Correction of Outliers in Temperature Time Series Based on Sliding Window Prediction in Meteorological Sensor Network

Correction of Outliers in Temperature Time Series Based on Sliding Window Prediction in Meteorological Sensor Network

Anomaly detection in server metrics with use of one-sided median algorithm

Automatic real-time fault detection for industrial assets using metasensors

Contact Info

Product

Resources

About