How can sliding HyperLogLog and EWMA detect port scan attacks in IP traffic?

Abstract

IP networks are constantly targeted by new techniques of denial of service attacks (SYN flooding, port scan, UDP flooding, etc), causing service disruption and considerable financial damage. The on-line detection of DoS attacks in the current high-bit rate IP traffic is a big challenge. We propose in this paper an on-line algorithm for port scan detection. It is composed of two complementary parts: First, a probabilistic counting part, where the number of distinct destination ports is estimated by adapting a method called 'sliding HyperLogLog' to the context of port scan in IP traffic. Second, a decisional mechanism is performed on the estimated number of destination ports in order to detect in real time any behavior that could be related to a malicious traffic. This latter part is mainly based on the exponentially weighted moving average algorithm (EWMA) that we adapted to the context of on-line analysis by adding a learning step (supposed without attacks) and improving its update mechanism. The obtained port scan detecting method is tested against real IP traffic containing some attacks. It detects all the port scan attacks within a very short time response (of about 30 s) and without any false positive. The algorithm uses a very small total memory of less than 22 kb and has a very good accuracy on the estimation of the number of destination ports (a relative error of about 3.25%), which is in agreement with the theoretical bounds provided by the sliding HyperLogLog algorithm.

Anomaly‐based intrusion detection systems: The requirements, methods, measurements, and datasets

“…Detecting online a change of the process parameters in data streams is an important issue as it can have several interpretations depending on the application domain. In Reference 166, the authors showed that an abrupt increase in the number of destination ports in IP traffic is an efficient criterion to detect port scan attacks. Given that the amount of internet and computer traffic is growing and increasing at a fast rate, monitoring the network in real‐time becomes a challenging task.…”

A Statistical Approach to Detect Jamming Attacks in Wireless Sensor Networks

“…EWMA, which was proposed by Roberts [ 33 ], is an efficient statistical technique used in detecting small shifts in time-series data. It functions by first defining a threshold that delimits a standard behaviour before periodically handling updates on average of the observed data traffic [ 34 ]. EWMA is also characterised with its low complexity because the weighted average only needs to be updated for each newly observed data.…”

“…In summary, our jamming detection technique consists of two phases; the first phase is the training phase that involves the capture of normal IAT from legitimate member nodes to the cluster head and also from the cluster heads to the base station to initialize its parameters and obtain a normal profile. Just like in the work of Chabchoub et al [ 34 ], no change point detection process will be performed during this phase. In the second phase, the test phase, a pattern change is detected during a jamming attack on a per packet basis using the EWMA algorithm.…”

Pay for a Sliding Bloom Filter and Get Counting, Distinct Elements, and Entropy for Free

“…HLL requires m bytes and its standard deviation is σ ≈ 1.04 √ m . SWHLL extends HLL to sliding windows [9,22], and was used to detect attacks such as port scans [10]. SWAMP's space requirement is proportional to W and thus, it is only comparable in space to HLL when ε −2 = O(W ).…”