Distributed, multi-level network anomaly detection for datacentre networks

Abstract

Over the past decade, numerous systems have been proposed to detect and subsequently prevent or mitigate security vulnerabilities. However, many existing intrusion or anomaly detection solutions are limited to a subset of the traffic due to scalability issues, hence failing to operate at line-rate on large, highspeed datacentre networks. In this paper, we present a two-level solution for anomaly detection leveraging independent execution and message passing semantics. We employ these constructs within a network-wide distributed anomaly detection framework that allows for greater detection accuracy and bandwidth cost saving through attack path reconstruction.Experimental results using real operational traffic traces and known network attacks generated through the Pytbull IDS evaluation framework, show that our approach is capable of detecting anomalies in a timely manner while allowing reconstruction of the attack path, hence further enabling the composition of advanced mitigation strategies. The resulting system shows high detection accuracy when compared to similar techniques, at least 20% better at detecting anomalies, and enables full path reconstruction even at smallto-moderate attack traffic intensities (as a fraction of the total traffic), saving up to 75% of bandwidth due to early attack detection.

State of the Art Literature Review on Network Anomaly Detection

“…The number of DDoS attacks are increasing due to the growing number of IoT devices with low security mechanisms and the fact that nowadays it is fairly easy to acquire attack tools. This has created a situation where large number of these devices can be used to perform distributed attacks, that is, to carry out more powerful attacks [1,2,5,7,12]. So far 24 different DDoS attack vectors have been found globally [12].…”

“…Therefore, any changes in flow can be detected by comparing network data to stored metrics and used to detect presence of anomalies. [2] The authors tested the proposed method for various attack types, such as Brute Force access, 0-day attacks and Port Scans. Their test results concluded that the solution was able to offer complete path reconstruction at the onset of DDoS attacks that generally have high intensity.…”

“…Also, partial path reconstruction was achieved for anomalies of lower intensity. [2] For future work M. Iordache et al mentioned an improvement for synchronizing remotely invoked instances by advanced majority voting scheme. The purpose of the improvement is to extend the amount of classification information what will be used for anomaly detection.…”

“…Secondly, they proposed further research in improving method deployment and management techniques by taking into account i) data locality, ii) system resource availability and demand and iii) guaranteeing network topology changes seamlessly. [2] The proposed method was intended for Data centres, but it can be used also in a local networks. When comparing to grid-based approximation, this solution is more comprehensive as it also offers also full path reconstruction and can detect other type of attacks.…”

High-performance, platform-independent DDoS detection for IoT ecosystems

“…In these approaches, coarse-grained detection is to be executed upstream in the network, closer to the attackers. However, this leads to the use of dedicated middleboxes scattered across the network for scrubbing purposes [9] [6]. For an IoT-DDoS detection solution (i.e., protecting the network against DDoS originated on IoT devices) to solve the above mentioned problem, it has to ensure: a) lightweight processing, by relying on traffic features and analysis methods targeting overhead minimization and coarse-grained anomaly detection; b) platform-independence, to minimize the need for purpose-built devices and the use of traffic redirectionbased approaches; and c) high-performance, in order to achieve fast reaction through early detection while avoiding performance degradation.…”