Payload-Byte: A Tool for Extracting and Labeling Packet Capture Files of Modern Network Intrusion Detection Datasets

Abstract

Adapting modern approaches for network intrusion detection is becoming critical, given the rapid technological advancement and adversarial attack rates. Therefore, packetbased methods utilizing payload data are gaining much popularity due to their effectiveness in detecting certain attacks. However, packet-based approaches suffer from a lack of standardization, resulting in incomparability and reproducibility issues. Unlike flow-based datasets, no standard labeled dataset exists, forcing researchers to follow bespoke labeling pipelines for individual approaches. Without a standardized baseline, proposed approaches cannot be compared and evaluated with each other. One cannot gauge whether the proposed approach is a methodological advancement or is just being benefited from the proprietary interpretation of the dataset. Addressing comparability and reproducibility issues, we introduce Payload-Byte, an open-source tool for extracting and labeling network packets in this work. Payload-Byte utilizes metadata information and labels raw traffic captures of modern intrusion detection datasets in a generalized manner. Moreover, we transformed the labeled data into a byte-wise feature vector that can be utilized for training machine learning models. The whole cycle of processing and labeling is explicitly stated in this work. Furthermore, source code and processed data are made publicly available so that it may act as a standardized baseline for future research work. Lastly, we present a brief comparative analysis of machine learning models trained on packet-based and flow-based data.

Graph representation learning for context-aware network intrusion detection

“…To handle this, we used Payload-Byte to extract and label the PCAP files of the two NIDS datasets mentioned above. 15 This tool takes raw PCAP data and parses the data into bytes and packet header features. Bytes that do not reach the default size of 1500 are padded with zeros.…”

“…The other features included are time-to-live, protocol, and time between packets. 15 As such, we are able to consider both packet and flow information in this work.…”

Enhancing IoT Security via Automatic Network Traffic Analysis: The Transition from Machine Learning to Deep Learning

“…The research in [37] and [9] influences our work, introducing a method for converting network traffic packets into images. By expending this innovative approach, we utilize a single DL model that autonomously learns features, resulting in significant time savings.…”

“…To address this challenge, researchers propose innovative techniques. For instance, in [38], [37], [9], the authors introduce novel methods to convert packets from PCAP files into images to use data representation techniques from the computer vision field for malware traffic classification. Similarly, NLP techniques are employed to extract features from textual descriptions like log files, as demonstrated in [19], [10], [12], [20].…”

Network intrusion detection system by learning jointly from tabular and text‐based features

“…Recognizing the growing need for standardized approaches in modern network intrusion detection datasets, Farrukh et al (2022) introduced Payload-Byte. This versatile tool streamlines dataset curation and establishes a standardized foundation for future research.…”