A Comparative Review of Security Threats Datasets for Vehicular Networks

Swessi, Dorsaf; Idoudi, Hanen

doi:10.1109/3ict53449.2021.9581683

Cited by 9 publications

(7 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the growing demand for datasets in vehicular network research, only a few surveys comprehensively review and compare the existing datasets. In their review of datasets in the context of V2X security, the authors in [ 15 ] analyzed and classified the datasets based on their targeted architecture, the types of attacks included in each dataset, and their severity. While their approach is commendable, their review of the existing datasets could be more exhaustive.…”

Section: Existing Surveys On Vehicular Network Datasetsmentioning

confidence: 99%

Investigation of Security Threat Datasets for Intra- and Inter-Vehicular Environments

Haddaji,

Ayed,

Chaari Fourati

et al. 2024

Sensors

View full text Add to dashboard Cite

Vehicular networks have become a critical component of modern transportation systems by facilitating communication between vehicles and infrastructure. Nonetheless, the security of such networks remains a significant concern, given the potential risks associated with cyberattacks. For this purpose, artificial intelligence approaches have been explored to enhance the security of vehicular networks. Using artificial intelligence algorithms to analyze large datasets can enable the early identification and mitigation of potential threats. However, developing and testing effective artificial-intelligence-based solutions for vehicular networks necessitates access to diverse datasets that accurately capture the various security challenges and attack scenarios in this context. In light of this, the present survey comprehensively examines the vehicular network environment, the associated security issues, and existing datasets. Specifically, we begin with a general overview of the vehicular network environment and its security challenges. Following this, we introduce an innovative taxonomy designed to classify datasets pertinent to vehicular network security and analyze key features of these datasets. The survey concludes with a tailored guide aimed at researchers in the vehicular network domain. This guide offers strategic advice on selecting the most appropriate datasets for specific research scenarios in the field.

show abstract

Section: Existing Surveys On Vehicular Network Datasetsmentioning

confidence: 99%

Investigation of Security Threat Datasets for Intra- and Inter-Vehicular Environments

Haddaji,

Ayed,

Chaari Fourati

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…ROAD also presents as a hub for researchers to reference the taxonomy of CAN data. Systematic and survey literature has recently been published citing the ROAD data [51,[80][81][82]. Some studies make claim that the ROAD dataset is the most comprehensive and realistic open CAN dataset available for evaluating and comparing CAN IDSs for attacks [51,81].…”

Section: Plos Onementioning

confidence: 99%

“…Systematic and survey literature has recently been published citing the ROAD data [51,[80][81][82]. Some studies make claim that the ROAD dataset is the most comprehensive and realistic open CAN dataset available for evaluating and comparing CAN IDSs for attacks [51,81]. Other research has referenced the quality of the dataset, used it to establish definitions within the CAN IDS research community, or cited the work as an establishment of research standards [21,[83][84][85][86][87][88][89][90].…”

Section: Plos Onementioning

confidence: 99%

A comprehensive guide to CAN IDS data and introduction of the ROAD dataset

Verma,

Bridges,

Iannacone

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Although ubiquitous in modern vehicles, Controller Area Networks (CANs) lack basic security properties and are easily exploitable. A rapidly growing field of CAN security research has emerged that seeks to detect intrusions or anomalies on CANs. Producing vehicular CAN data with a variety of intrusions is a difficult task for most researchers as it requires expensive assets and deep expertise. To illuminate this task, we introduce the first comprehensive guide to the existing open CAN intrusion detection system (IDS) datasets. We categorize attacks on CANs including fabrication (adding frames, e.g., flooding or targeting and ID), suspension (removing an ID’s frames), and masquerade attacks (spoofed frames sent in lieu of suspended ones). We provide a quality analysis of each dataset; an enumeration of each datasets’ attacks, benefits, and drawbacks; categorization as real vs. simulated CAN data and real vs. simulated attacks; whether the data is raw CAN data or signal-translated; number of vehicles/CANs; quantity in terms of time; and finally a suggested use case of each dataset. State-of-the-art public CAN IDS datasets are limited to real fabrication (simple message injection) attacks and simulated attacks often in synthetic data, lacking fidelity. In general, the physical effects of attacks on the vehicle are not verified in the available datasets. Only one dataset provides signal-translated data but is missing a corresponding “raw” binary version. This issue pigeon-holes CAN IDS research into testing on limited and often inappropriate data (usually with attacks that are too easily detectable to truly test the method). The scarcity of appropriate data has stymied comparability and reproducibility of results for researchers. As our primary contribution, we present the Real ORNL Automotive Dynamometer (ROAD) CAN IDS dataset, consisting of over 3.5 hours of one vehicle’s CAN data. ROAD contains ambient data recorded during a diverse set of activities, and attacks of increasing stealth with multiple variants and instances of real (i.e. non-simulated) fuzzing, fabrication, unique advanced attacks, and simulated masquerade attacks. To facilitate a benchmark for CAN IDS methods that require signal-translated inputs, we also provide the signal time series format for many of the CAN captures. Our contributions aim to facilitate appropriate benchmarking and needed comparability in the CAN IDS research field.

show abstract

“…A masquerade attack can be seen as a combination of a suspension attack and a spoofing attack. An attacker suspends message transmission from a specific ECU and transmits manipulated messages at the same frequency to the CAN bus to maintain the original CAN busload [31]. Lee et al [30] first published masquerade attack datasets performed on real vehicles by exploiting diagnostic services and a bus-off attack.…”

Section: Masquerade Attackmentioning

confidence: 99%

“…Despite these advancements, there is a need to evaluate the quality of these published datasets to ensure their effectiveness and suitability for IDS training and evaluation purposes. However, existing studies have predominantly focused on qualitative aspects, overlooking the quantitative measures of attack dataset quality [29,31,32]. We demonstrate the limitations of currently available published datasets for automotive ML-based IDS to illustrate the importance of high-quality datasets.…”

Section: Introductionmentioning

confidence: 97%

A Comprehensive Analysis of Datasets for Automotive Intrusion Detection Systems

Lee,

Choi,

Kim

et al. 2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Recently, automotive intrusion detection systems (IDSs) have emerged as promising defense approaches to counter attacks on in-vehicle networks (IVNs). However, the effectiveness of IDSs relies heavily on the quality of the datasets used for training and evaluation. Despite the availability of several datasets for automotive IDSs, there has been a lack of comprehensive analysis focusing on assessing these datasets. This paper aims to address the need for dataset assessment in the context of automotive IDSs. It proposes qualitative and quantitative metrics that are independent of specific automotive IDSs, to evaluate the quality of datasets. These metrics take into consideration various aspects such as dataset description, collection environment, and attack complexity. This paper evaluates eight commonly used datasets for automotive IDSs using the proposed metrics. The evaluation reveals biases in the datasets, particularly in terms of limited contexts and lack of diversity. Additionally, it highlights that the attacks in the datasets were mostly injected without considering normal behaviors, which poses challenges for training and evaluating machine learning-based IDSs. This paper emphasizes the importance of addressing the identified limitations in existing datasets to improve the performance and adaptability of automotive IDSs. The proposed metrics can serve as valuable guidelines for researchers and practitioners in selecting and constructing high-quality datasets for automotive security applications. Finally, this paper presents the requirements for high-quality datasets, including the need for representativeness, diversity, and balance.

show abstract

A Comparative Review of Security Threats Datasets for Vehicular Networks

Cited by 9 publications

References 32 publications

Investigation of Security Threat Datasets for Intra- and Inter-Vehicular Environments

Investigation of Security Threat Datasets for Intra- and Inter-Vehicular Environments

A comprehensive guide to CAN IDS data and introduction of the ROAD dataset

A Comprehensive Analysis of Datasets for Automotive Intrusion Detection Systems

Contact Info

Product

Resources

About