DARN: Dynamic Baselines for Real-time Network Monitoring

Mijumbi, Rashid; Asthana, Abhaya; Koivunen, Markku; Haiyong, Fu; Norman, Zhu

doi:10.1109/netsoft.2018.8460047

Cited by 6 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main contributions of this paper are threefold: (1) a machine learning‐based algorithm that automatically generates network baselines, (2) an analytics approach that adapts the generated baselines to changes in network conditions, and (3) a learning algorithm that allows the system to learn from baseline breaches and metric similarities to minimize false alarms. This paper is an extension of an earlier version 4 with the following improvements: (1) a background discussion on LSTM has been added, (2) we have improved the discussion of related work, (3) we have compared the prediction accuracy of LSTM with other state‐of‐the‐art time series prediction techniques, (4) we have added an example to demonstrate the effect of expert feedback, and (5) we have added a flowchart to summarize the steps involved in the proposed approaches.…”

Section: Introductionmentioning

confidence: 93%

Design, implementation, and evaluation of learning algorithms for dynamic real‐time network monitoring

et al. 2020

Self Cite

View full text Add to dashboard Cite

Network monitoring is necessary so as to ensure high reliability and availability in telecom networks. One of the main challenges posed by state‐of‐the‐art monitoring tools is the creation of network baselines. Such baselines include thresholds that can be used to determine whether monitored values (with a given context, e.g., time) represent normal network operation or not. The size and complexity of current (and future) networks make it infeasible to manually determine and set baselines for each network operator and metric, let alone adapting the thresholds to changes in network conditions. This leads to the use of default baselines and/or setting baselines only once and never changing them throughout the lifetime of network elements. This does not only cause inefficient operation but could have implications for network reliability and availability. In this paper, we present the design, implementation, and evaluation of DARN: a collection of analytics and machine learning‐based algorithms aimed at ensuring that network baselines are automatically adapted to different metric evolution. DARN has been comprehensively evaluated on a deployment with real traffic to confirm accuracy of generated baselines, a 22% improvement in accuracy due to baseline adaptation and a 72% reduction in false alarms.

show abstract

Section: Introductionmentioning

confidence: 93%

Design, implementation, and evaluation of learning algorithms for dynamic real‐time network monitoring

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Simulations show that it can allow reducing by 50% the duration of the training phase of the LSTM NN. LSTM NNs have been used to adapt network baseline estimation to changes in cloud environments as well, as described in [35]. Authors propose to create a network baseline through LSTM AEs, adapting it when metric trend changes.…”

Section: Deep Lstm-based Autoencodersmentioning

confidence: 99%

LSTM-based radiography for anomaly detection in softwarized infrastructures

Diamanti

Vílchez

Secci

2020

2020 32nd International Teletraffic Congress (ITC 32)

View full text Add to dashboard Cite

Legacy and novel network services are expected to be migrated and designed to be deployed in fully virtualized environments. Starting with 5G, NFV becomes a formally required brick in the specifications, for services integrated within the infrastructure provider networks. This evolution leads to deployment of virtual resources Virtual-Machine (VM)-based, container-based and/or server-less platforms, all calling for a deep virtualization of infrastructure components. Such a network softwarization also unleashes further logical network virtualization, easing multi-layered, multi-actor and multi-access services, so as to be able to fulfill high availability, security, privacy and resilience requirements. However, the derived increased components heterogeneity makes the detection and the characterization of anomalies difficult, hence the relationship between anomaly detection and corresponding reconfiguration of the NFV stack to mitigate anomalies. In this article we propose an unsupervised machine-learning data-driven approach based on Long-Short-Term-Memory (LSTM) autoencoders to detect and characterize anomalies in virtualized networking services. With a radiography visualization, this approach can spot and describe deviations from nominal parameter values of any virtualized network service by means of a lightweight and iterative mean-squared reconstruction error analysis of LSTM-based autoencoders. We implement and validate the proposed methodology through experimental tests on a vIMS proof-of-concept deployed using Kubernetes.

show abstract

“…We consider in this work, as an illustrative example, a monitoring variable that represents the CPU utilization of a pollee node. In the state space, the quality Θ is defined as good when the monitored CPU utilization is under a fixed threshold, which we set to 60% as suggested in [5]. If the monitored attribute value crosses this threshold, an anomaly is considered and has to be detected by the poller node.…”

Section: A Simulation Setupmentioning

confidence: 99%

“…Wuhib et al [4] proposed an adaptive aggregation protocol of monitoring data to detect threshold crossings, where nodes dynamically adjust their neighbor interaction rates which considerably reduce communication overhead. Mijumbi et al [5] developed DARN, a real-time monitoring system using machine learning and neural networks. The goal of DARN is to classify the monitoring metrics into clusters and then automatically generates and adapts the lower and upper baselines of the changes in network operating conditions.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Reinforcement Learning for Adaptive Monitoring of Low-Power IoT Networks

Frikha

Lahmadi

Gammar

et al. 2020

2020 16th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)

View full text Add to dashboard Cite

Low-power Internet of Things (IoT) networks are widely deployed in various environments with resource constrained devices, making their states monitoring particularly challenging. In this paper, we propose an adaptive monitoring mechanism for low-power IoT devices, by using a reinforcement learning (RL) method to automatically adapt the polling frequencies of the collected attributes. Our goal is to minimize the number of monitoring packets while keeping accurate and timely detection of threshold crossings associated to supervised attributes. We study the various RL parameter settings under different monitoring attribute behaviors using OpenAi Gym simulator. We implement the RL based adaptive polling in Contiki OS and we evaluate its performance using Cooja simulator. Our results show that our approach converges to optimal polling frequencies and outperforms static periodic notification-based methods by reducing the number of monitoring packets, with a percentage of correctly detected threshold crossings exceeding 80%.

show abstract

DARN: Dynamic Baselines for Real-time Network Monitoring

Cited by 6 publications

References 15 publications

Design, implementation, and evaluation of learning algorithms for dynamic real‐time network monitoring

Design, implementation, and evaluation of learning algorithms for dynamic real‐time network monitoring

LSTM-based radiography for anomaly detection in softwarized infrastructures

Leveraging Reinforcement Learning for Adaptive Monitoring of Low-Power IoT Networks

Contact Info

Product

Resources

About