The steady improvement of advanced driving assistance systems (ADAS) and the leap towards automated driving (AD) require novel methods for assessing the safety of those, which is a major subject for current research. Different proposals cope with the massive testing effort to assure the safety of such systems. These proposals include virtualization of testing, usage of stochastic methods and reduction of the necessary real world driving tests. Despite these different approaches, they all rely on the same basis: The behavior assessment of the vehicle under test, which results in a measurement of risk.This paper presents a novel approach to measure the criticality of a given driving scenario fitted on the requirements of testing. A Monte-Carlo simulation, which uses the input of a motion prediction model as variation parameters, determines the possible evolutions of a scenario at every time step. The distributions of these parameters have been fitted to data obtained by a large-scale field tests. These evolutions are then analyzed individually by considering the Time-To-React (TTR) measure. Finally a single value of accident risk between 0 and 1 can be assigned to the scenario.

Assessment and testing are among the biggest challenges for the release of automated driving. Up to this date, the exact procedure to achieve homologation is not settled. Current research focuses on scenario-based approaches that represent driving scenarios as test cases within a scenario space. This avoids redundancies in testing, enables the inclusion of virtual testing into the process, and makes a statement about test coverage possible. However, it is unclear how to define such a scenario space and the coverage criterion.This work presents a novel approach to the definition of the scenario space. Spatiotemporal filtering on naturalistic highway driving data provides a large amount of driving scenarios as a foundation. A custom distance measure between scenarios enables hierarchical agglomerative clustering, categorizing the scenarios into subspaces. The members of a resulting cluster found through this approach reveal a common structure that is visually observable. We discuss a data-driven solution to define the necessary test coverage for the assessment of automated driving. Finally, the contribution of the findings to achieve homologation is elaborated.

<div class="section abstract"><div class="htmlview paragraph">A key criterion for launching autonomous vehicles on real roads is the knowledge of their capability to ensure traffic safety. In contrast to ADAS, deriving this measure of safety is difficult to achieve as the functional scope of an autonomous driving function exceeds by far the one of ADAS. As a consequence, real-world testing solely is not sufficient enough to cover the required test volume. This assessment problem imposes new requirements on a valid test concept for automated driving. A possible solution represents simulation by enabling it to generate reliable test kilometers. As a first step, we discuss in this paper the feasibility of simulation frameworks to re-simulate a real-world test in certain scenarios. We will demonstrate that even with ground truth information of the vehicle odometry and corresponding environment model an acceptable accordance of functional behavior is not guaranteed. Hence, to yield a reliable degree of confidence in a risk assessment a single scenario has to be represented by an ensemble generated from a local variation considering both, ground truth information and odometry including the environment model. In order to achieve these statements we first introduce a valid representation of traffic scenarios acting as a test case description for an autonomous driving function. Afterwards, the description based on the vehicle odometry and created environment model as well as the description based on the ground truth measured via Differential GPS are re-simulated using the same autonomous driving function as deployed in the test vehicle. The reprocessed traces are compared to the corresponding real-world data to illustrate resulting behavior changes in the autonomous driving function. To make the behavior changes interpretable for the assessment process a sensitive risk value is deployed containing information about the reprocessing quality of the chosen description and simulation.</div></div>

No abstract

One remaining challenge for Automated Driving (AD) that remains unclear to this day is its assessment for market release. The application of previous strategies derived from the V-model is infeasible due to the vast amount of required real-road testing to prove safety with an acceptable significance. A full set of requirements covering all possible traffic scenarios for testing and AD system can still not be derived to this day. Several approaches address this issue by either improving the set of test cases or by including other virtual test domains in the assessment process. However, all rely on simulations that can not be validated as a whole and therefore not be used for proving safety.This work addresses this issue and exhibits a method to verify the use of simulation in a scenario-based assessment process. By introducing a pipeline for reprocessing real-world scenarios as test cases we demonstrate where errors emerge and how these can be isolated. We unveil an issue in simulation which may cause behavior changes of the AD function in resimulation and thus makes the straight forward use of simulation in the assessment process impossible. A solution promising to minimize reprocessing errors and to avoid this behavior change is presented. Finally, this enables the local variation of realworld driving tests in a solely simulative context yielding verified and usable results.