Unsupervised Anomaly Event Detection for VNF Service Monitoring Using Multivariate Online Arima

Schmidt, Florian; Suri-Payer, Florian; Gulenko, Anton; Wallschläger, Marcel; Acker, Alexander; Kao, Odej

doi:10.1109/cloudcom2018.2018.00061

Cited by 15 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…References Time series analysis [162], [161], [76], [198], [83], [71], [210], [70] Bayesian learning [118], [85], [14], [103], [159], [50], [35], [128], [105], [157] Principal component analysis [195], [5], [18], [113], [108] Regression analysis [205], [54], [25], [89], [156] Logistic regression [131], [115], [26], [207] Hidden Markov model [79], [78], [11], [170] Markov chain [167], [52], [41] Gaussian mixture models [106], [13] Statistical tests [173], [114] Restricted Boltzmann machine [110], [ [201], [200], [63], Wavelet transform…”

Section: Methodsmentioning

confidence: 99%

A Systematic Review on Anomaly Detection for Cloud Computing Environments

Hagemann

Katsarou

2020

2020 3rd Artificial Intelligence and Cloud Computing Conference

View full text Add to dashboard Cite

The detection of anomalies in data is a far-reaching field of research which also applies to the field of cloud computing in several different ways: from the detection of various types of intrusions to the detection of hardware failures, many publications address how far anomaly detection methods are able to meet the specific requirements of a cloud-based network. Since there is still no comprehensive overview of this constantly growing field of research, this literature review provides a systematic evaluation of 215 publications that can be considered as representative for the last ten years of this scientific development. Our analysis identifies three main methodological areas (machine learning, deep learning, statistical approaches) and summarizes how exactly the corresponding models are applied for the detection of anomalies. Furthermore, we clarify which concrete application areas are typically addressed by anomaly detection in the context of cloud computing environments and which related public datasets are often used for evaluations. Finally, we discuss the implications of the literature review and provide directions for future research.

show abstract

Section: Methodsmentioning

confidence: 99%

A Systematic Review on Anomaly Detection for Cloud Computing Environments

Hagemann

Katsarou

2020

2020 3rd Artificial Intelligence and Cloud Computing Conference

View full text Add to dashboard Cite

show abstract

“…The implementation of the anomaly detection algorithms follow the principle of Identity Function and Threshold Model [12] to automatically adjust the anomaly detection model to the evolving data stream in an unsupervised manner. We integrated Long-short term memory (LSTM) [12], BIRCH [13] and Autoregressive integrated moving average (ARIMA) [14] as reconstruction functions, while applying exponential moving average as dynamic threshold model. As anomalies might propagate between monitored components, we apply a time-based root cause analysis.…”

Section: B Data Analysismentioning

confidence: 99%

Towards AIOps in Edge Computing Environments

Becker,

Schmidt,

Gulenko

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Edge computing was introduced as a technical enabler for the demanding requirements of new network technologies like 5G. It aims to overcome challenges related to centralized cloud computing environments by distributing computational resources to the edge of the network towards the customers. The complexity of the emerging infrastructures increases significantly, together with the ramifications of outages on critical use cases such as self-driving cars or health care. Artificial Intelligence for IT Operations (AIOps) aims to support human operators in managing complex infrastructures by using machine learning methods. This paper describes the system design of an AIOps platform which is applicable in heterogeneous, distributed environments. The overhead of a high-frequency monitoring solution on edge devices is evaluated and performance experiments regarding the applicability of three anomaly detection algorithms on edge devices are conducted. The results show, that it is feasible to collect metrics with a high frequency and simultaneously run specific anomaly detection algorithms directly on edge devices with a reasonable overhead on the resource utilization.

show abstract

“…Where α 0 is constant, φ is autoregressive coefficients, θ is moving average and ε t is white noise at time t , and L is the lag operator which when applied to Y returns the prior value. The choice of ARIMA was attributable for it scalability in related work (Yu et al, 2016;Wang et al, 2016;Ahmar et al, 2018;Calheiros et al, 2015;Schmidt et al, 2018). Moreover, it is specifically built and works well for time series data (Hyndman and Athanasopoulos, 2018).…”

Section: Prediction Toolmentioning

confidence: 99%

Fast Analysis and Prediction in Large Scale Virtual Machines Resource Utilisation

Abubakar

Barbhuiya

Kilpatrick

et al. 2020

Proceedings of the 10th International Conference on Cloud Computing and Services Science

View full text Add to dashboard Cite

Most Cloud providers running Virtual Machines (VMs) have a constant goal of preventing downtime, increasing performance and power management among others. The most effective way to achieve these goals is to be proactive by predicting the behaviours of the VMs. Analysing VMs is important, as it can help cloud providers gain insights to understand the needs of their customers, predict their demands, and optimise the use of resources. To manage the resources in the cloud efficiently, and to ensure the performance of cloud services, it is crucial to predict the behaviour of VMs accurately. This will also help the cloud provider improve VM placement, scheduling, consolidation, power management, etc. In this paper, we proposed a framework for fast analysis and prediction in large scale VM CPU utilisation. We use a novel approach both in terms of the algorithms employed for prediction and in terms of the tools used to run these algorithms with a large dataset to deliver a solid VM CPU utilisation predictor. We processed over two million VMs from Microsoft Azure VM traces and filter out the VMs with complete one month of data which amount to 28,858VMs. The filtered VMs were subsequently used for prediction. Our Statistical analysis reveals that 94% of these VMs are predictable. Furthermore, we investigate the patterns and behaviours of those VMs and realised that most VMs have one or several spikes of which majority of those spikes are not seasonal. For all the 28,858VMs analysed and forecasted, we accurately predicted 17,523 (61%) VMs based on their CPU. We use Apache Spark for parallel and distributed processing to achieve fast processing. In term of fast processing (execution time), on average, each VM is analysed and predicted within three seconds.

show abstract

Unsupervised Anomaly Event Detection for VNF Service Monitoring Using Multivariate Online Arima

Cited by 15 publications

References 12 publications

A Systematic Review on Anomaly Detection for Cloud Computing Environments

A Systematic Review on Anomaly Detection for Cloud Computing Environments

Towards AIOps in Edge Computing Environments

Fast Analysis and Prediction in Large Scale Virtual Machines Resource Utilisation

Contact Info

Product

Resources

About