Network anomaly detection using a cross‐correlation‐based long‐range dependence analysis

Abstract

The detection of anomalies in network traffic is an important task in today's Internet. Among various anomaly detection methods, the techniques based on examination of the long-range dependence (LRD) behavior of network traffic stands out to be powerful. In this paper, we reveal anomalies in aggregated network traffic by examining the LRD behavior based on the cross-correlation function of the bidirectional control and data planes traffic. Specifically, observing that the conventional cross-correlation function has a low measure of dissimilarity between the two planes, which leads to a reduced anomaly detection performance, we propose a modification of the cross-correlation function to mitigate this issue. The performance of the proposed method is analyzed using a relatively recent Internet traffic captured at King Saud University. The results demonstrate that using the modified cross-correlation function has the ability to detect low volume and short duration attacks. It also compensates for some misdetections exhibited by using the autocorrelation structures of the bidirectional traffic of the control, data, and WHOLE (combined control and data) planes traffic. 1 | INTRODUCTION Internet services have been rapidly evolving due to continuous emergence of new technologies and high bandwidth applications. According to Cisco, the annual global IP traffic is expected to reach 3.30 Zettabyte by 2021. 1 This growth has introduced many challenges in terms of securing information, reliability of data transfer, and detection of traffic anomalies and attacks. Anomalies are patterns of data that deviate from normal behavior. A broad variety of such patterns in Internet traffic are frequently faced by network operators. These patterns could represent genuine abnormalities induced by physical or technical issues, such as high-rate flows, network outage, and sudden changes due to flash crowds. 2 However, such patterns could also be generated by malicious activities such as cyber intrusions, distributed denial of service (DDoS) attacks, port scanning, and worm propagation. 3,4 This type of malicious activities could likely introduce disastrous effects on the proper operation of networks. The mentioned malicious activities are considerably growing with time. According to the worldwide infrastructure security report, 5 it is evident that the volume of these attacks on data centers, networks infrastructures, and

Challenges in Identifying Network Attacks Using Netflow Data

“…Data binning is a preprocessing step for presenting the network flow events in a form on which analysis algorithms can be applied. Binning the network events have been applied as a preprocessing step to detect low volume and short duration attacks [7] and separate a set of network traffic measurements that correspond to normal and abnormal behaviour [27]. A time-bin is defined as a window of one fixed time interval.…”

“…Their method used Principal Component Analysis to identify normal and abnormal network conditions by using a set of network traffic measurements. In [7], the authors modified the cross-correlation function to improve the anomaly detection performance of the conventional cross-correlation function. In [4], the authors setup shallow Convolution Neural Network (CNN), moderate CNN and deep CNN to assess enhancements for proactive attack handling.…”

“…To better support the processes of attack mitigation, it is helpful to first understand how an attack transpires in practice. Recent work that analyzed the network logs or process logs for anomaly detection [3]- [7] and malicious authentication detection [8]- [10] have revealed useful insights to address attacks in networks. The time elapsed between the precursor event and an attack is defined as the lead time.…”

“…Analyzing attacks in any large-scale or complex network require awareness of the sequence of events that is encountered by the network components. While researchers have focused on specific components [7], [10] depending on their target problem, answering how attacks occur needs a more integrated approach towards correlation-based log-mining [13]. Our work is novel in that it considers the flow connection-specific events along with their inter-component relationships to increase the lead times to identification of an attack boosting network attack prediction schemes.…”

“…However, it lacks the broader view. For example, analyzing the network routers [7] w.r.t. the hosts separately may miss anomalies in the event logs generated elsewhere.…”