Methods for Improving the Accuracy of the Virtual Assessment of Autonomous Driving

Notz, Dominik; Sigl, Martin; Kühbeck, Thomas; Wagner, Sebastian; Groh, Korbinian; Schütz, Christoph; Watzenig, Daniel

doi:10.1109/iccve45908.2019.8965040

Cited by 10 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The collected data can be used for a variety of purposes. As suggested in [8] models that try to imitate human driving behavior can be trained on our realistic TO trajectory data. Such models can be used in simulators and significantly increase their realism.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, TO trajectory datasets enable the training of smart agents that imitate human behavior. Smart agent models can be used in simulation and re-processing to increase their accuracy and validity [8].…”

Section: Introductionmentioning

confidence: 99%

“…Hence, to increase the coverage of the traffic space of human driving behavior in a dataset we need to passively record traffic. In particular, infrastructure sensors seem suitable for this task [8].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Extraction and Assessment of Naturalistic Human Driving Trajectories from Infrastructure Camera and Radar Sensors

Notz¹,

Stanke²,

Kühbeck³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Collecting realistic driving trajectories is crucial for training machine learning models that imitate human driving behavior. Most of today's autonomous driving datasets contain only a few trajectories per location and are recorded with test vehicles that are cautiously driven by trained drivers. In particular in interactive scenarios such as highway merges, the test driver's behavior significantly influences other vehicles. This influence prevents recording the whole traffic space of human driving behavior. In this work, we present a novel methodology to extract trajectories of traffic objects using infrastructure sensors. Infrastructure sensors allow us to record a lot of data for one location and take the test drivers out of the loop. We develop both a hardware setup consisting of a camera and a traffic surveillance radar and a trajectory extraction algorithm. Our vision pipeline accurately detects objects, fuses camera and radar detections and tracks them over time. We improve a state-of-the-art object tracker by combining the tracking in image coordinates with a Kalman filter in road coordinates. We show that our sensor fusion approach successfully combines the advantages of camera and radar detections and outperforms either single sensor. Finally, we also evaluate the accuracy of our trajectory extraction pipeline. For that, we equip our test vehicle with a differential GPS sensor and use it to collect ground truth trajectories. With this data we compute the measurement errors. While we use the mean error to de-bias the trajectories, the error standard deviation is in the magnitude of the ground truth data inaccuracy. Hence, the extracted trajectories are not only naturalistic but also highly accurate and prove the potential of using infrastructure sensors to extract real-world trajectories.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Extraction and Assessment of Naturalistic Human Driving Trajectories from Infrastructure Camera and Radar Sensors

Notz¹,

Stanke²,

Kühbeck³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Graphical comparisons and validation metrics such as the correlation coefficient were used at different stages of the AV pipeline. The influence of ground truth measurements compared to environment sensor data on the model validation of the entire AV is analyzed in [74,132,187]. They use multiple box plots to visualize how simulation errors flow through the AV pipeline.…”

Section: Deterministic Simulationsmentioning

confidence: 99%

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Riedmaier

Danquah

Schick

et al. 2020

Arch Computat Methods Eng

View full text Add to dashboard Cite

Simulation is becoming increasingly important in the development, testing and approval process in many areas of engineering, ranging from finite element models to highly complex cyber-physical systems such as autonomous cars. Simulation must be accompanied by model verification, validation and uncertainty quantification (VV&UQ) activities to assess the inherent errors and uncertainties of each simulation model. However, the VV&UQ methods differ greatly between the application areas. In general, a major challenge is the aggregation of uncertainties from calibration and validation experiments to the actual model predictions under new, untested conditions. This is especially relevant due to high extrapolation uncertainties, if the experimental conditions differ strongly from the prediction conditions, or if the output quantities required for prediction cannot be measured during the experiments. In this paper, both the heterogeneous VV&UQ landscape and the challenge of aggregation will be addressed with a novel modular and unified framework to enable credible decision making based on simulation models. This paper contains a comprehensive survey of over 200 literature sources from many application areas and embeds them into the unified framework. In addition, this paper analyzes and compares the VV&UQ methods and the application areas in order to identify strengths and weaknesses and to derive further research directions. The framework thus combines a variety of VV&UQ methods, so that different engineering areas can benefit from new methods and combinations. Finally, this paper presents a procedure to select a suitable method from the framework for the desired application.

show abstract

“…However, this raises the question on the credibility of the tests performed in the simulation. It cannot be denied that a gap exists in relation to real world test drives [4], [5], [6], [7], cf. Fig.…”

Section: Introductionmentioning

confidence: 99%

A Credibility Assessment Approach for Scenario-Based Virtual Testing of Automated Driving Functions

Stadler¹,

Montanari²,

Baron

et al. 2022

IEEE Open J. Intell. Transp. Syst.

View full text Add to dashboard Cite

An immense test space is pushing the development and testing of automated driving functions from real to virtual environments. The virtual world is provided by interconnected simulation models representing sensors, vehicle dynamics, and both static and dynamic environment. For the virtual validation of automated driving, special attention must be paid to the simulation's credibility, which can be impaired by inappropriate or inaccurate simulation models and tools. Therefore, in this work a method is proposed to assess the credibility of simulation-based testing for automated driving. The approach allows a qualitative and relatively quantitative comparisons between scenarios as well as between different simulation setups. Therefore, several uni-and multivariate metrics are applied towards a scoring of similarity of the behavior between simulation and real test drive. This is achieved by using ground truth data in form of simulation scenarios from real world measurement data. In this way, the virtual automated vehicle encounters the same conditions and surroundings than its counterpart in the real world for evaluating their similarity. The practical applicability of the proposed credibility assessment approach is demonstrated in a case study, in which the credibility of an exemplary simulation-based test bench is inferred.

show abstract

Methods for Improving the Accuracy of the Virtual Assessment of Autonomous Driving

Cited by 10 publications

References 15 publications

Extraction and Assessment of Naturalistic Human Driving Trajectories from Infrastructure Camera and Radar Sensors

Extraction and Assessment of Naturalistic Human Driving Trajectories from Infrastructure Camera and Radar Sensors

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

A Credibility Assessment Approach for Scenario-Based Virtual Testing of Automated Driving Functions

Contact Info

Product

Resources

About