Safe At Any Speed: A Simulation-Based Test Harness for Autonomous Vehicles

Abbas, Houssam; O’Kelly, Matthew; Rodionova, Alëna; Mangharam, Rahul

doi:10.1007/978-3-030-17910-6_8

Cited by 18 publications

(21 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related work has already been published by the author in [134], [135]. Gangopadhyay et al [136] use a Bayesian optimization, [137] use a random forest model and Abbas et al [138] use simulated annealing in their test harness capable of testing perception algorithms.…”

Section: Simulation-based Falsificationmentioning

confidence: 99%

Survey on Scenario-Based Safety Assessment of Automated Vehicles

et al. 2020

View full text Add to dashboard Cite

When will automated vehicles come onto the market? This question has puzzled the automotive industry and society for years. The technology and its implementation have made rapid progress over the last decade, but the challenge of how to prove the safety of these systems has not yet been solved. Since a market launch without proof of safety would neither be accepted by society nor by legislators, much time and many resources have been invested into safety assessment in recent years in order to develop new approaches for an efficient assessment. This paper therefore provides an overview of various approaches, and gives a comprehensive survey of the so-called scenario-based approach. The scenario-based approach is a promising method, in which individual traffic situations are typically tested by means of virtual simulation. Since an infinite number of different scenarios can theoretically occur in real-world traffic, even the scenario-based approach leaves the question unanswered as to how to break these down into a finite set of scenarios, and find those which are representative in order to render testing more manageable. This paper provides a comprehensive literature review of related safety-assessment publications that deal precisely with this question. Therefore, this paper develops a novel taxonomy for the scenario-based approach, and classifies all literature sources. Based on this, the existing methods will be compared with each other and, as one conclusion, the alternative concept of formal verification will be combined with the scenario-based approach. Finally, future research priorities are derived.

show abstract

Section: Simulation-based Falsificationmentioning

confidence: 99%

Survey on Scenario-Based Safety Assessment of Automated Vehicles

et al. 2020

View full text Add to dashboard Cite

show abstract

“…However, this requires a probabilistic representation. Assuming there are uncertain inputs and parameters and a total error e resulting from (3) and (4). Then a variance decomposition based on the law of total variance [48,Eq.…”

Section: Sources Of Errors and Uncertaintiesmentioning

confidence: 99%

“…They also use algorithms on real and synthetic images and eight different metrics to assess the realto-virtual performance gap. Abbas et al [3] similarly assess image datasets of their driving simulator and additionally compare the visual complexity regarding color and spatial information. Similar to those camera approaches, Jasinski [95] validates radar sensor models.…”

Section: Deterministic Simulationsmentioning

confidence: 99%

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Riedmaier

Danquah

Schick

et al. 2020

Arch Computat Methods Eng

View full text Add to dashboard Cite

Simulation is becoming increasingly important in the development, testing and approval process in many areas of engineering, ranging from finite element models to highly complex cyber-physical systems such as autonomous cars. Simulation must be accompanied by model verification, validation and uncertainty quantification (VV&UQ) activities to assess the inherent errors and uncertainties of each simulation model. However, the VV&UQ methods differ greatly between the application areas. In general, a major challenge is the aggregation of uncertainties from calibration and validation experiments to the actual model predictions under new, untested conditions. This is especially relevant due to high extrapolation uncertainties, if the experimental conditions differ strongly from the prediction conditions, or if the output quantities required for prediction cannot be measured during the experiments. In this paper, both the heterogeneous VV&UQ landscape and the challenge of aggregation will be addressed with a novel modular and unified framework to enable credible decision making based on simulation models. This paper contains a comprehensive survey of over 200 literature sources from many application areas and embeds them into the unified framework. In addition, this paper analyzes and compares the VV&UQ methods and the application areas in order to identify strengths and weaknesses and to derive further research directions. The framework thus combines a variety of VV&UQ methods, so that different engineering areas can benefit from new methods and combinations. Finally, this paper presents a procedure to select a suitable method from the framework for the desired application.

show abstract

“…In [16], a test harness is presented that allows an AV to drive in a simulated world in real-time, as illustrated in Figure 5. A notable aspect of this harness is that it allows weather conditions to vary, thus stressing the perception pipeline.…”

Section: About the Authorsmentioning

confidence: 99%

“…This is very important: the 2016 fatal accident in Florida involving a Tesla Auto-Pilot was partially due to a failure of the car's visual sensors to detect the truck blocking the AV's path against the bright sky. Issues like validity of simulated data are also addressed in [16].…”

Section: About the Authorsmentioning

confidence: 99%

A Driver’s License Test for Driverless Vehicles

Abbas

O’Kelly

Rodionova

et al. 2017

Mechanical Engineering

Self Cite

View full text Add to dashboard Cite

This article elaborates the approaches that can be used to verify an autonomous vehicle (AV) before giving it a driver’s license. Formal methods applied to the problem of AV verification include theorem proving, reachability analysis, synthesis, and maneuver design. Theorem proving is an interactive technique in which the computer is largely responsible for demonstrating that the model satisfies the specification, with occasional help from the user. The latter provides lemmas and axioms that the tool leverages to advance the proof towards its conclusion. Reachability analysis is used to verify the operation of the AV during navigation. This provides an extension of onboard diagnostics to whole-AV operation, where the diagnosis does not concern one component’s requirements, but the safety of the entire AV. Another approach is to design correct-by-construction controllers from preverified maneuvers. The basic idea is that one builds a library of maneuvers, such as Left-Turn and Right-Turn, and verifies that the car can perform these maneuvers from any initial state.

show abstract

Safe At Any Speed: A Simulation-Based Test Harness for Autonomous Vehicles

Cited by 18 publications

References 10 publications

Survey on Scenario-Based Safety Assessment of Automated Vehicles

Survey on Scenario-Based Safety Assessment of Automated Vehicles

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

A Driver’s License Test for Driverless Vehicles

Contact Info

Product

Resources

About