A Review of Time-Series Anomaly Detection Techniques: A Step to Future Perspectives

Shaukat, Kamran; Alam, Talha Mahboob; Luo, Siwei; Shabbir, Shakir; Hameed, Ibrahim A.; Li, Jiaming; Abbas, Syed Konain; Javed, Umair

doi:10.1007/978-3-030-73100-7_60

Cited by 67 publications

(25 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Shaukat pada [15] menyatakan bahwa menentukan rentang data normal dalam membangun sistem merupakan sebuah tantangan dalam membangun sistem deteksi anomali pada data time series. Penelitian terkait deteksi kesalahan data telah dilakukan oleh wibowo pada [16].…”

Section: Pendahuluanunclassified

Deteksi Anomali Konduktivitas Air Menggunakan Kalman Filter

Putra

Koprawi

Ashari

et al. 2022

Buletin Ilmiah Sarjana Teknik Elektro

View full text Add to dashboard Cite

Water quality is an essential part of shrimp farming. Data integrity is one of the challenges in building a water conductivity monitoring system. Data read by the sensor should represent the physical conditions that occur. However, some factors can cause abnormal data changes. This abnormal data change can occur due to sensor damage or an attempt to sabotage the pool. In this study, a data anomaly detection algorithm was built using the Kalman filter and standard deviation to solve the problem of determining the normal range of data. The designed algorithm was then tested and evaluated using Arduino nano, Arduino mega, and Wemos D1 Microcontrollers to determine the algorithm's performance on limited computing devices. Based on the data analysis that has been carried out, it is found that the anomaly detection algorithm based on the Kalman filter has an accuracy of 92.5% and can detect anomaly data that occurs with TPF = 1 and FNR = 0 values. The implementation of the detection algorithm on the microcontroller shows that WEMOS D1 (ESP8266) has an excellent average computational speed of 27.99 us. As for the stability of the Arduino Nano (ATMEGA328) and Arduino Mega 2560 (ATMEGA 2560) microcontrollers, the computation time deviation is about 2.8 us. Kualitas air merupakan bagian penting pada budidaya udang. Salah satu tantangan dalam membangun sebuah sistem monitoring konduktivitas air adalah Keutuhan data. Suatu data yang terbaca oleh sensor seharusnya mewakili kondisi fisik yang terjadi. Akan tetapi ada faktor-faktor dapat menyebabkan perubahan data yang tidak wajar. Perubahan data yang tidak wajar ini dapat terjadi karena disebabkan kerusakan sensor maupun adanya upaya sabotase pada kolam. Pada penelitian ini dibangun sebuah algoritma deteksi anomali data menggunakan Kalman filter dan standar deviasi untuk mengatasi masalah penentuan rentang data normal. Algoritma yang dirancang kemudian diuji dan dievaluasi dengan menggunakan Mikrokontroller Arduino nano, Arduino mega dan Wemos D1 untuk mengetahui performa algoritma yang dirancang pada perangkat komputasi terbatas. Berdasarkan analisis data yang telah dilakukan didapatkan hasil bahwa algoritma deteksi anomali berbasis kalman filter memiliki akurasi 92,5% dan dapat mendeteksi data anomali yang terjadi dengan nilai TPF =1 dan FNR=0. Implementasi algoritma deteksi pada mikrokontroller menunjukkan bahwa WEMOS D1 (ESP8266) memiliki rata-rata kecepatan komputasi yang baik yaitu 27,99 us. Sedangkan untuk kestabilan mikrokontroller Arduino Nano (ATMEGA328) dan Arduino Mega 2560 (ATMEGA 2560) memiliki deviasi waktu komputasi sekitar 2,8 us.

show abstract

Section: Pendahuluanunclassified

Deteksi Anomali Konduktivitas Air Menggunakan Kalman Filter

Putra

Koprawi

Ashari

et al. 2022

Buletin Ilmiah Sarjana Teknik Elektro

View full text Add to dashboard Cite

show abstract

“…ML techniques have widely used in the domain of education [13,14], software measurement [15][16][17], decision support system [18,19], social sciences [20,21], healthcare [22][23][24], and disease diagnosis [9,25]. Numerous computational methods were used in the renewable energy domain [26][27][28][29][30].…”

Section: Introductionmentioning

confidence: 99%

A Machine Learning-Based Model for Stability Prediction of Decentralized Power Grid Linked with Renewable Energy Resources

Ibrar

Hassan

Shaukat

et al. 2022

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

A decentralized power grid is a modern system that implements demand response without requiring major infrastructure changes. In decentralization, the consumers regulate their electricity demand autonomously based on the grid frequency. With cheap equipment (i.e., smart meters), the grid frequency can be easily measured anywhere. Electrical grids need to be stable to balance electricity supply and demand to ensure economically and dynamically viable grid operation. The volumes of electricity consumed/produced (p) by each grid participant, cost-sensitivity (g), and grid participants’ response times (tau) to changing grid conditions affect the stability of the grid. Renewable energy resources are volatile on varying time scales. Due to the volatile nature of these renewable energy resources, there are more frequent fluctuations in decentralized grids integrating renewable energy resources. The decentralized grid is designed by linking real-time electricity rates to the grid frequency over a few seconds to provide demand-side control. In this study, a model has been proposed to predict the stability of a decentralized power grid. The simulated data obtained from the online machine learning repository has been employed. Data normalization has been employed to reduce the biased behavior among attributes. Various data level resampling techniques have been used to address the issue of data imbalance. The results showed that a balanced dataset outperformed an imbalanced dataset regarding classifiers’ performance. It has also been observed that oversampling techniques proved better than undersampling techniques and imbalanced datasets. Overall, the XGBoost algorithm outperformed all other machine learning algorithms based on performance. XGBoost has been given an accuracy of 94.7%, but while combining XGBoost with random oversampling, its accuracy prediction has been improved to 96.8%. This model can better predict frequency fluctuations in decentralized power grids and the volatile nature of renewable energy resources resulting in better utilization. This prediction may contribute to the stability of a decentralized power grid for better distribution and management of electricity.

show abstract

“…Designing a time series forecast for a particular use case typically incorporates five sections. The first section of the design process is the data preprocessing to transform the raw data into a desirable form for the forecasting method (Shaukat et al, 2021; Wang & Wang, 2020). The second section is feature engineering, which aims to extract hidden characteristics of the considered time series or to identify useful exogenous information for the forecasting method (Zebari et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…In this context, the long‐term objective toward full automation has motivated numerous researchers and led to promising research results in the fields related to this review study, as shown in Figure 2. Several surveys and review studies analyze the automation of single forecasting pipeline sections such as preprocessing (Shaukat et al, 2021; Wang & Wang, 2020), feature engineering (Zebari et al, 2020), HPO and forecasting method selection (Hutter et al, 2019; Zöller & Huber, 2021), and forecast ensembling (Hajirahimi & Khashei, 2019). Moreover, rather than focusing on the automated design of the entire forecasting pipeline, existing studies on time series forecasting only consider the statistical or machine learning forecasting methods themselves (De Gooijer & Hyndman, 2006; Han et al, 2019; Taieb et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

Review of automated time series forecasting pipelines

Meisenbacher

Turowski

Phipps

et al. 2022

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Time series forecasting is fundamental for various use cases in different domains such as energy systems and economics. Creating a forecasting model for a specific use case requires an iterative and complex design process. The typical design process includes five sections (1) data preprocessing, (2) feature engineering, (3) hyperparameter optimization, (4) forecasting method selection, and (5) forecast ensembling, which are commonly organized in a pipeline structure. One promising approach to handle the ever‐growing demand for time series forecasts is automating this design process. The article, thus, reviews existing literature on automated time series forecasting pipelines and analyzes how the design process of forecasting models is currently automated. Thereby, we consider both automated machine learning (AutoML) and automated statistical forecasting methods in a single forecasting pipeline. For this purpose, we first present and compare the identified automation methods for each pipeline section. Second, we analyze these automation methods regarding their interaction, combination, and coverage of the five pipeline sections. For both, we discuss the reviewed literature that contributes toward automating the design process, identify problems, give recommendations, and suggest future research. This review reveals that the majority of the reviewed literature only covers two or three of the five pipeline sections. We conclude that future research has to holistically consider the automation of the forecasting pipeline to enable the large‐scale application of time series forecasting. This article is categorized under: Technologies > Machine Learning Technologies > Prediction Algorithmic Development > Spatial and Temporal Data Mining

show abstract

A Review of Time-Series Anomaly Detection Techniques: A Step to Future Perspectives

Cited by 67 publications

References 37 publications

Deteksi Anomali Konduktivitas Air Menggunakan Kalman Filter

Deteksi Anomali Konduktivitas Air Menggunakan Kalman Filter

A Machine Learning-Based Model for Stability Prediction of Decentralized Power Grid Linked with Renewable Energy Resources

Review of automated time series forecasting pipelines

Contact Info

Product

Resources

About