Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control

Abstract

Deep neural network controllers for autonomous driving have recently benefited from significant performance improvements, and have begun deployment in the real world. Prior to their widespread adoption, safety guarantees are needed on the controller behaviour that properly take account of the uncertainty within the model as well as sensor noise. Bayesian neural networks, which assume a prior over the weights, have been shown capable of producing such uncertainty measures, but properties surrounding their safety have not yet been quantified for use in autonomous driving scenarios. In this paper, we develop a framework based on a state-of-theart simulator for evaluating end-to-end Bayesian controllers. In addition to computing pointwise uncertainty measures that can be computed in real time and with statistical guarantees, we also provide a method for estimating the probability that, given a scenario, the controller keeps the car safe within a finite horizon. We experimentally evaluate the quality of uncertainty computation by three Bayesian inference methods in different scenarios and show how the uncertainty measures can be combined and calibrated for use in collision avoidance. Our results suggest that uncertainty estimates can greatly aid decision making in autonomous driving.

Robustness of Bayesian Neural Networks to Gradient-Based Attacks

“…We discuss reasons these methods are fooled under our framework in Sections 5 and 6. Statistical techniques for the quantification of adversarial robustness of BNNs have been introduced by Cardelli et al (2019a) and employed in (Michelmore et al, 2019) to detect erroneous behaviours in the context of autonomous driving. Furthermore, in (Ye & Zhu, 2018) a Bayesian approach has been considered in the context of adversarial training, where the authors showed improved performances with respect to other, non-Bayesian, adversarial training approaches.…”

Reliability Analysis of Artificial Intelligence Systems Using Recurrent Events Data from Autonomous Vehicles

“…Åsljung et al (2017) used extreme value theory to model the safety of AV. Michelmore et al (2019) designed a statistical framework to evaluate the safety of deep neural network controllers and assessed the safety of AV. Burton et al (2020) provided a multidisciplinary perspective on the safety of AV from engineering, ethics and law aspects.…”

Trajectory Planning Under Environmental Uncertainty With Finite-Sample Safety Guarantees

“…In either case, a large number of samples (e.g. potential positions of obstacles) is made available, leading to a requirement for an approach that reasons about the magnitude of the uncertainty based on these samples and ensures safety while allowing for real-time computation [20]. The approach proposed in this work addresses this issue of generating safe trajectories while working with an arbitrary finite number of samples.…”