Automotive Cybersecurity Testing: Survey of Testbeds and Methods

Abstract

Computing and connectivity capabilities in modern cars have introduced new cybersecurity challenges that can potentially affect the safety of an automobile and its occupants. Effective cybersecurity testing of vehicles can play a crucial role in discovering and addressing security flaws; however testing a real vehicle (involving cyber-physical components) carries safety and economic risks. Therefore, many researchers and practitioners rely on testing environments (commonly known as testbeds) for uncovering cybersecurity vulnerabilities. Effective and efficient security testing needs the application of appropriate and systematic testing methods. This study presents a survey of seven different automotive cybersecurity testbeds proposed over the last ten years (between 2012 and 2019) as well as four different types of cybersecurity testing methods employed by cybersecurity researchers. This survey will help students, researchers, and professionals with designing and building their testbeds and refining their testing techniques and methodologies.

Survey of Model-Based Security Testing Approaches in the Automotive Domain

“…As outlined in Section I, a main drawbacks of the current security testing process is the complexity of modern vehicles, late and manual testing techniques, such as penetration testing, and the challenge of identifying vulnerabilities as early as possible through testing. Existing surveys on (model-based) security testing, such as the work of Mahmood et al [63], Luo et al [64], and Pekaric et al [65] often refer to penetration testing and dynamic analysis techniques (e.g., fuzzing and vulnerability scanning) that do not support early vulnerability testing and can only be applied late in the development cycle. Surveys, such as the work of Altinger et al [15] and Kriebel et al [17], suggest that model-based testing addresses these challenges.…”

“…Mahmood et al [63] present a survey of seven test beds for security testing in the automotive field. The respective publications are examined and compared with regard to covered network technologies (e.g., CAN), attack surfaces, and attack techniques.…”

CANLay: A Network Virtualized Testbed for Vehicle Systems – Improving System Integration and Verification Efforts

“…Previous research has established two critical criteria for quality evaluation of automotive networking test beds: fidelity and adaptability [6]. Fidelity refers to the capacity to emulate real-world in-vehicle networking infrastructure, while adaptability concerns the ability to simulate various real-world infrastructures.…”

CANNON: Reliable and Stealthy Remote Shutdown Attacks via Unaltered Automotive Microcontrollers

“…There has been some effort in addressing these issues with running software simulations of ECUs and networks on the cloud. However, these are often extremely limited and do not represent even a fraction of all attack surfaces of a modern vehicle [12]. More importantly, any hardwarebased weaknesses and vulnerabilities are often completely absent in software-only simulations.…”