Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Tuncali, Cumhur Erkan; Fainekos, Georgios; Ito, Hideto; Kapinski, James

doi:10.1109/ivs.2018.8500421

Cited by 185 publications

(117 citation statements)

References 36 publications

Supporting

Mentioning

116

Contrasting

Unclassified

Order By: Relevance

“…More recently, adversarial example attacks have been implemented in the physical domain [2], [6], [21], such as adding stickers to a stop sign that result in misclassification [2]. However, these attacks still focus on perception models disembodied from the target application, such as autonomous driving, and few efforts study such attacks deployed directly on dynamical systems, such as autonomous driving [22], [23].…”

Section: Attacks On Deep Learning For Perception and Controlmentioning

confidence: 99%

Attacking vision-based perception in end-to-end autonomous driving models

Boloor

Garimella

et al. 2020

Journal of Systems Architecture

View full text Add to dashboard Cite

Recent advances in machine learning, especially techniques such as deep neural networks, are enabling a range of emerging applications. One such example is autonomous driving, which often relies on deep learning for perception. However, deep learning-based perception has been shown to be vulnerable to a host of subtle adversarial manipulations of images. Nevertheless, the vast majority of such demonstrations focus on perception that is disembodied from end-to-end control. We present novel endto-end attacks on autonomous driving in simulation, using simple physically realizable attacks: the painting of black lines on the road. These attacks target deep neural network models for endto-end autonomous driving control. A systematic investigation shows that such attacks are easy to engineer, and we describe scenarios (e.g., right turns) in which they are highly effective. We define several objective functions that quantify the success of an attack and develop techniques based on Bayesian Optimization to efficiently traverse the search space of higher dimensional attacks. Additionally, we define a novel class of hijacking attacks, where painted lines on the road cause the driverless car to follow a target path. Through the use of network deconvolution, we provide insights into the successful attacks, which appear to work by mimicking activations of entirely different scenarios. Our code is available on https://github.com/xz-group/AdverseDrive Index Terms-machine learning, adversarial examples, autonomous driving, end-to-end learning, bayesian optimization

show abstract

Section: Attacks On Deep Learning For Perception and Controlmentioning

confidence: 99%

Attacking vision-based perception in end-to-end autonomous driving models

Boloor

Garimella

et al. 2020

Journal of Systems Architecture

View full text Add to dashboard Cite

show abstract

“…In [17] and [18], test case generation for automated vehicles is realized using S-TaLiRo. In [19], an approach using S-TaLiRo is applied to motion planners based on machine learning, including simulated camera processing using deep neural networks.…”

Section: A Related Workmentioning

confidence: 99%

Generating Critical Test Scenarios for Automated Vehicles with Evolutionary Algorithms

Klischat

Althoff

2019

2019 IEEE Intelligent Vehicles Symposium (IV)

124

View full text Add to dashboard Cite

Virtual testing of automated vehicles using simulations is essential during their development. When it comes to the testing of motion planning algorithms, one is mainly interested in challenging, critical scenarios for which it is hard to find a feasible solution. However, these situations are rare under usual traffic conditions, demanding an automatic generation of critical test scenarios. We present an approach that automatically generates critical scenarios based on a minimization of the solution space of the vehicle under test. By formulating a scenario parametrization and automatic determination of relevant parameter intervals, we are able to optimize the criticality of complex scenarios. We use evolutionary algorithms to tackle the resulting highly nonlinear optimization problem. Compared to our previous approach, we are now able to handle complex situations, in particular those involving intersections. Finally, we demonstrate our approach by generating critical scenarios from initially uncritical scenarios.All authors are with the Technische Universität München,

show abstract

“…However, existing investigations on adversarial examples still focus on classification errors associated with static images and are conducted in limited experimental environments [3], [5], [11]. Research considering the learning model in a dynamic system setting, like on autonomous vehicles in the real world is sparse [12]. In this paper, we aim to address these current limitations and provide a methodology to systematically study physically realizable attacks on the e2e models in realistic driving conditions.…”

Section: B Attacks On Deep Learning For Perception and Controlmentioning

confidence: 99%

Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models

Boloor

Gill

et al. 2019

2019 IEEE International Conference on Embedded Software and Systems (ICESS)

View full text Add to dashboard Cite

Recent advances in machine learning, especially techniques such as deep neural networks, are promoting a range of high-stakes applications, including autonomous driving, which often relies on deep learning for perception. While deep learning for perception has been shown to be vulnerable to a host of subtle adversarial manipulations of images, end-toend demonstrations of successful attacks, which manipulate the physical environment and result in physical consequences, are scarce. Moreover, attacks typically involve carefully constructed adversarial examples at the level of pixels. We demonstrate the first end-to-end attacks on autonomous driving in simulation, using simple physically realizable attacks: the painting of black lines on the road. These attacks target deep neural network models for end-to-end autonomous driving control. A systematic investigation shows that such attacks are surprisingly easy to engineer, and we describe scenarios (e.g., right turns) in which they are highly effective, and others that are less vulnerable (e.g., driving straight). Further, we use network deconvolution to demonstrate that the attacks succeed by inducing activation patterns similar to entirely different scenarios used in training.

show abstract

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Cited by 185 publications

References 36 publications

Attacking vision-based perception in end-to-end autonomous driving models

Attacking vision-based perception in end-to-end autonomous driving models

Generating Critical Test Scenarios for Automated Vehicles with Evolutionary Algorithms

Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models

Contact Info

Product

Resources

About