Efficient Splitting of Test and Simulation Cases for the Verification of Highly Automated Driving Functions

Böde, Eckard; Büker, Matthias; Eberle, Ulrich; Fränzle, Martin; Gerwinn, Sebastian; Kramer, Birte

doi:10.1007/978-3-319-99130-6_10

Cited by 10 publications

(16 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Böde et al [36] aim to find an optimal split between the number of scenarios in simulation and the real world. They decompose the probability that the system satisfies its requirements into two terms based on the law of total probability.…”

Section: Scenario Assignment Methodsmentioning

confidence: 99%

“…The current state-of-the-art regarding the assignment of scenarios to test environments focuses either on a scenario-independent level of the entire use case [35] or on the number of test scenarios based on budget constraints [36,39]. However, this is not sufficient to select how concrete validation scenarios shall be distributed across the scenario space.…”

Section: Analysis Of the Literaturementioning

confidence: 99%

See 1 more Smart Citation

Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles

et al. 2020

View full text Add to dashboard Cite

Due to the rapid progress in the development of automated vehicles over the last decade, their market entry is getting closer. One of the remaining challenges is the safety assessment and type approval of automated vehicles, as conventional testing in the real world would involve an unmanageable mileage. Scenario-based testing using simulation is a promising candidate for overcoming this approval trap. Although the research community has recognized the importance of safeguarding in recent years, the quality of simulation models is rarely taken into account. Without investigating the errors and uncertainties of models, virtual statements about vehicle safety are meaningless. This paper describes a whole process combining model validation and safety assessment. It is demonstrated by means of an actual type-approval regulation that deals with the safety assessment of lane-keeping systems. Based on a thorough analysis of the current state-of-the-art, this paper introduces two approaches for selecting test scenarios. While the model validation scenarios are planned from scratch and focus on scenario coverage, the type-approval scenarios are extracted from measurement data based on a data-driven pipeline. The deviations between lane-keeping behavior in the real and virtual world are quantified using a statistical validation metric. They are then modeled using a regression technique and inferred from the validation experiments to the unseen virtual type-approval scenarios. Finally, this paper examines safety-critical lane crossings, taking into account the modeling errors. It demonstrates the potential of the virtual-based safeguarding process using exemplary simulations and real driving tests.

show abstract

Section: Scenario Assignment Methodsmentioning

confidence: 99%

Section: Analysis Of the Literaturementioning

confidence: 99%

Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles

et al. 2020

View full text Add to dashboard Cite

show abstract

“…3 "SV" in the original text. Böde et al [33] dealt with the question of what an optimal (with respect to the resulting test case execution costs) tradeoff between test case execution in simulation and driving tests might look like. In their method, test case execution in simulation is used to verify the safety properties of the (highly) automated driving function; driving tests are used to validate the simulation models used.…”

Section: B Test Case Assignment Methodsmentioning

confidence: 99%

“…There are methods for assigning test cases to simulative test methods (Schuldt et al [32] and Schuldt [6]), for splitting the execution of test cases into simulation and reality (Böde et al [33]), which is a kind of test case assignment, and recommendations for choosing among four test methods (Antkiewicz et al [34]). However, assigning test cases to test methods is not expedient because a test method alone does not provide enough information about the actual performance of a specific test bench configuration used to execute a test case (see Section III-B).…”

mentioning

confidence: 99%

Toward Generating Sufficiently Valid Test Case Results: A Method for Systematically Assigning Test Cases to Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles

Steimle

Weber²

2022

IEEE Access

View full text Add to dashboard Cite

“…Mullins et al [127] select calibration and validation scenarios so that costs and prediction uncertainties are minimized. Böde et al [27] select validation and application scenarios based on a similar concept.…”

Section: Scenario Designmentioning

confidence: 99%

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Riedmaier

Danquah

Schick

et al. 2020

Arch Computat Methods Eng

View full text Add to dashboard Cite

Simulation is becoming increasingly important in the development, testing and approval process in many areas of engineering, ranging from finite element models to highly complex cyber-physical systems such as autonomous cars. Simulation must be accompanied by model verification, validation and uncertainty quantification (VV&UQ) activities to assess the inherent errors and uncertainties of each simulation model. However, the VV&UQ methods differ greatly between the application areas. In general, a major challenge is the aggregation of uncertainties from calibration and validation experiments to the actual model predictions under new, untested conditions. This is especially relevant due to high extrapolation uncertainties, if the experimental conditions differ strongly from the prediction conditions, or if the output quantities required for prediction cannot be measured during the experiments. In this paper, both the heterogeneous VV&UQ landscape and the challenge of aggregation will be addressed with a novel modular and unified framework to enable credible decision making based on simulation models. This paper contains a comprehensive survey of over 200 literature sources from many application areas and embeds them into the unified framework. In addition, this paper analyzes and compares the VV&UQ methods and the application areas in order to identify strengths and weaknesses and to derive further research directions. The framework thus combines a variety of VV&UQ methods, so that different engineering areas can benefit from new methods and combinations. Finally, this paper presents a procedure to select a suitable method from the framework for the desired application.

show abstract

Efficient Splitting of Test and Simulation Cases for the Verification of Highly Automated Driving Functions

Cited by 10 publications

References 6 publications

Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles

Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles

Toward Generating Sufficiently Valid Test Case Results: A Method for Systematically Assigning Test Cases to Test Bench Configurations in a Scenario-Based Test Approach for Automated Vehicles

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Contact Info

Product

Resources

About