A Reference Dataset for Network Traffic Activity Based Intrusion Detection System

Abstract

Abstract:The network traffic dataset is a crucial part of anomaly based intrusion detection systems (IDSs). These IDSs train themselves to learn normal and anomalous activities. Properly labeled dataset is used for the training purpose. For the activities based IDSs, proper network traffic activity labeled dataset is the first requirement, however non-availability of such datasets is bottlenecked in the field of IDS research. In this experiment, a synthetic dataset "Panjab University -Intrusion Dataset (PU-IDS)" is created. The purpose of this study is to provide the researchers a reference dataset for the performance evaluation of network traffic activity based IDSs. University of New Brunswick Network Security Laboratory -Knowledge Disscovery in Databases (NSL-KDD) is a benchmark dataset for anomaly detection but it does not contain activity based labeling. So basic characteristics of this dataset are taken for the generation of the new synthetic dataset with various activities based labels. The dataset is first categorized as per protocol and service. Thereafter, as per minimum & maximum values of attributes, activity profiles are synthetically generated. This paper also discusses various statistical characteristics of PU-IDS. The total number of 198533 instances along with 273 of activity profiles are created. This dataset also contain different 98 protocol_service profiles.

A survey of network-based intrusion detection data sets

“…ISCX 2012 [28] Four attack scenarios (1: Infiltrating the network from the inside; 2: HTTP DoS; 3: DDoS using an IRC botnet; 4: SSH brute force) ISOT [57] botnet (Storm, Waledac) KDD CUP 99 [42] DoS, privilege escalation (remote-to-local and user-to-root), probing Kent 2016 [58], [59] not specified Kyoto 2006+ [60] Various attacks against honeypots (e.g. backscatter, DoS, exploits, malware, port scans, shellcode) LBNL [61] port scans NDSec-1 [62] botnet (Citadel), brute force (against FTP, HTTP and SSH), DDoS (HTTP floods, SYN flooding and UDP floods), exploits, probe, spoofing, SSL proxy, XSS/SQL injection NGIDS-DS [19] backdoors, DoS, exploits, generic, reconnaissance, shellcode, worms NSL-KDD [63] DoS, privilege escalation (remote-to-local and user-to-root), probing PU-IDS [64] DoS, privilege escalation (remote-to-local and user-to-root), probing PUF [65] DNS attacks SANTA [35] (D)DoS (ICMP flood, RUDY, SYN flood), DNS amplification, heartbleed, port scans SSENET-2011 [47] DoS (executed through LOIC), port scans (executed through Angry IP Scanner, Nessus, Nmap), various attack tools (e.g. metasploit) SSENET-2014 [66] botnet, flooding, privilege escalation, port scans SSHCure [67] SSH attacks TRAbID [68] DoS (HTTP flood, ICMP flood, SMTP flood, SYN flood, TCP keepalive), port scans (ACK-Scan, FIN-Scan, NULL-Scan, OS Fingerprinting, Service Fingerprinting, UDP-Scan, XMAS-Scan) TUIDS [69], [70] botnet (IRC), DDoS (Fraggle flood, Ping flood, RST flood, smurf ICMP flood, SYN flood, UDP flood), port scans (e.g.…”

“…The authors developed a generator which extracts statistics of an input data set and uses these statistics to generate new synthetic instances. As a consequence, the work of Singh et al [64] could be seen as a traffic generator to create PU-IDS which contains about 200,000 data points and has the same attributes and format as the NSL-KDD data set. As NSL-KDD is based on KDD CUP 1999 which in turn is extracted from DARPA 1998, the year of creation is set to 1998 since the input for the traffic generator was captured back then.…”

AI-IDS: Application of Deep Learning to Real-Time Web Intrusion Detection

“…3) PU-IDS DATASETS PU-IDS [24] is a derivative of the NSL-KDD data set, and its author has developed a generator that extracts the statistics of the input data set and then creates a new data set. A traffic generator has the same attributes and format as the NSL-KDD data set.…”

Towards Generating Benchmark Datasets for Worm Infection Studies

“…KDD Cup [17], NSL-KDD [18] and PU-IDS [19] are outdated, and they are not suitable for today's evaluations because of the many modifications that have occurred in the traffic of networks so far.…”