Have it Your Way: Generating Customized Log Datasets With a Model-Driven Simulation Testbed

Landauer, Max; Skopik, Florian; Wurzenberger, Markus; Hotwagner, Wolfgang; Rauber, Andreas

doi:10.1109/tr.2020.3031317

Cited by 29 publications

(36 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the following section, we analyze existing testbeds and show that all of them have major flaws with regard to the presented requirements, thus impeding their utility for the generation of log data for sound cybersecurity experiments. LARIAT [51] National Cyber Range [13] CyberVAN [10] ViSe [50] DETERLab [5] ATT&CK Evaluations [63] DetectionLab [31] SimuLand [37] Skopik et al [57] Landauer et al [27] SOCBED (this paper)…”

Section: Testbeds For Artifact Generationmentioning

confidence: 99%

“…However, the system's source code is not publicly available and no self-tests are included. Finally, Landauer et al [27] introduce the concept of a model-driven testbed generator, together with an implementation for web applications. They do provide a dataset created using the testbed, but neither the testbed nor its generator are publicly available, and hence, our requirements are not met.…”

Section: Analysis Of Related Workmentioning

confidence: 99%

“…However, this does not imply that each iteration of our experiment (and thus SOCBED) produces the exact same log data (highlighted by the differences for attack steps (1) and (3) in Table 2). Such differences result from an inherent trade-off between realism and replicability when using virtual machines for log data generation and can be attributed to different effects such as background processes and time-dependent tasks [27,57]. We further analyze the impact of such variations in Section 6.6 and discuss resulting limitations in Section 7.…”

Section: Soundness Of the Experimentsmentioning

confidence: 99%

“…While we have shown that our exemplary experiment is reproducible, this does not imply that each iteration of SOCBED generates the exact same set of log data. Various factors such as background processes and time-dependent tasks influence the generated log data, resulting in variations between iterations and, possibly, also host computers [27,57]. This is an unavoidable trade-off between realism and replicability when generating log data using real systems.…”

Section: Deep Dive: Variations In Log Datamentioning

confidence: 99%

See 3 more Smart Citations

Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments

Uetz

Hemminghaus

Hackländer

et al. 2021

Annual Computer Security Applications Conference

View full text Add to dashboard Cite

Artifacts such as log data and network traffic are fundamental for cybersecurity research, e.g., in the area of intrusion detection. Yet, most research is based on artifacts that are not available to others or cannot be adapted to own purposes, thus making it difficult to reproduce and build on existing work. In this paper, we identify the challenges of artifact generation with the goal of conducting sound experiments that are valid, controlled, and reproducible. We argue that testbeds for artifact generation have to be designed specifically with reproducibility and adaptability in mind. To achieve this goal, we present SOCBED, our proof-of-concept implementation and the first testbed with a focus on generating realistic log data for cybersecurity experiments in a reproducible and adaptable manner. SOCBED enables researchers to reproduce testbed instances on commodity computers, adapt them according to own requirements, and verify their correct functionality. We evaluate SOCBED with an exemplary, practical experiment on detecting a multi-step intrusion of an enterprise network and show that the resulting experiment is indeed valid, controlled, and reproducible. Both SOCBED and the log dataset underlying our evaluation are freely available. CCS CONCEPTS• Security and privacy → Intrusion/anomaly detection and malware mitigation; Network security; • Computing methodologies → Modeling and simulation.

show abstract

Section: Testbeds For Artifact Generationmentioning

confidence: 99%

Section: Analysis Of Related Workmentioning

confidence: 99%

Section: Soundness Of the Experimentsmentioning

confidence: 99%

Section: Deep Dive: Variations In Log Datamentioning

confidence: 99%

See 2 more Smart Citations

Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments

Uetz

Hemminghaus

Hackländer

et al. 2021

Annual Computer Security Applications Conference

View full text Add to dashboard Cite

show abstract

“…Previous work on generating and labeling data synthetically also exists [43]- [47]. Synthetic techniques suffer to various degrees with artificial artifacts, but issues with unrealistic data have been generally accepted in light of the working assumption that real data on real networks is too cumbersome to generate and too sensitive to provide to researchers.…”

Section: Related Workmentioning

confidence: 99%

On Generating and Labeling Network Traffic with Realistic, Self-Propagating Malware

Buchanan¹,

Collyer²,

Davidson³

et al. 2021

Preprint

View full text Add to dashboard Cite

Research and development of techniques which detect or remediate malicious network activity require access to diverse, realistic, contemporary data sets containing labeled malicious connections. In the absence of such data, said techniques cannot be meaningfully trained, tested, and evaluated. Synthetically produced data containing fabricated or merged network traffic is of limited value as it is easily distinguishable from real traffic by even simple machine learning (ML) algorithms. Real network data is preferable, but while ubiquitous is broadly both sensitive and lacking in ground truth labels, limiting its utility for ML research.This paper presents a multi-faceted approach to generating a data set of labeled malicious connections embedded within anonymized network traffic collected from large production networks. Real-world malware is defanged and introduced to simulated, secured nodes within those networks to generate realistic traffic while maintaining sufficient isolation to protect real data and infrastructure. Network sensor data, including this embedded malware traffic, is collected at a network edge and anonymized for research use.Network traffic was collected and produced in accordance with the aforementioned methods at two major educational institutions. The result is a highly realistic, long term, multiinstitution data set with embedded data labels spanning over 1.5 trillion connections and over a petabyte of sensor log data. The usability of this data set is demonstrated by its utility to our artificial intelligence and machine learning (AI/ML) research program.

show abstract

Iterative Selection of Categorical Variables for Log Data Anomaly Detection

Landauer

Höld

Wurzenberger

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Have it Your Way: Generating Customized Log Datasets With a Model-Driven Simulation Testbed

Cited by 29 publications

References 20 publications

Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments

Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments

On Generating and Labeling Network Traffic with Realistic, Self-Propagating Malware

Iterative Selection of Categorical Variables for Log Data Anomaly Detection

Contact Info

Product

Resources

About