Validation of vehicle environment sensor models

Schaermann, Alexander; Rauch, Andreas; Hirsenkorn, Nils; Hanke, Timo; Rasshofer, R.H.; Biebl, Erwin

doi:10.1109/ivs.2017.7995752

Cited by 28 publications

(23 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of the metrics still have to be interpreted by one or more "experts", though it is not clear what qualifies to be one. Additionally, [8] shows a comparison of real and synthetic lidar data in a static scenario is shown. The sensor data is compared on occupancy grid level using quantitative metrics.…”

Section: Related Workmentioning

confidence: 99%

“…Thereby, a paradox appears in this context, as typically no stakeholder has the complete overview and knowledge about the required and the possible fidelity of sensor models, yet, which are generally not identical. While [8] is directly related to the work in the following, progress here is provided by benchmarking different models in different simulation tools. In fact, this enables writing requirements and interpreting metrics from the insight into lidar sensor systems and from first benchmarks of different modeling approaches and interfaces.…”

Section: Related Workmentioning

confidence: 99%

“…The immense effort e.g. for modeling static objects in adequate precision like 5 cm deviation in geometric dimensions of the models themselves and their geo-referenced position is described in [8]. In the scope of this work, static scenarios are used in which no environment is modeled and only cars are considered to avoid complications caused by differences of the environment simulation compared to reality, as stated in Sec.…”

Section: Environment Modelingmentioning

confidence: 99%

“…For the second category of metrics, two-dimensional occupancy grids are generated from the point clouds in a first step. When comparing different occupancy grids, various metrics can be found in the literature, like the overall error or the Barons cross correlation coefficient described in [8]. The occupied cells ratio (OCR) metric is chosen here, which is based on a cell-wise comparison of the real and simulated occupancy grid.…”

Section: A Metrics On Point Cloud Interface (If2)mentioning

confidence: 99%

See 3 more Smart Citations

Benchmarking and Functional Decomposition of Automotive Lidar Sensor Models

Rosenberger

Holder

Hueh

et al. 2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Simulation-based testing is seen as a major requirement for the safety validation of highly automated driving. One crucial part of such test architectures are models of environment perception sensors such as camera, lidar and radar sensors. Currently, an objective evaluation and the comparison of different modeling approaches for automotive lidar sensors are still a challenge. In this work, a real lidar sensor system used for object recognition is first functionally decomposed. The resulting sequence of processing blocks and interfaces is then mapped onto simulation methods. Subsequently, metrics applied to the aforementioned interfaces are derived, enabling a quantitative comparison between simulated and real sensor data at different steps of the processing pipeline. Benchmarks for several existing sensor models at a concrete selected interface are performed using those metrics by comparing them to measurements gained from the real sensor. Finally, we outline how metrics on low-level interfaces can correlate with results on more abstract ones. A major achievement of this work lies within the commonly accepted interfaces and a common understanding of real and virtual lidar sensor systems and, even more important, an initial guideline for the quantitative comparison of sensor models with the ambition to support future validation of virtual sensor models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Environment Modelingmentioning

confidence: 99%

Section: A Metrics On Point Cloud Interface (If2)mentioning

confidence: 99%

See 2 more Smart Citations

Benchmarking and Functional Decomposition of Automotive Lidar Sensor Models

Rosenberger

Holder

Hueh

et al. 2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

show abstract

“…For implementation we propose to use state of the art ray tracing, e.g. like in [11], to apply all the aspects we will describe in the following chapters. The desired solution after all would be a model leveling out the existing dilemma between efficient simulation and high fidelity [12].…”

Section: Introductionmentioning

confidence: 99%

Analysis of Real World Sensor Behavior for Rising Fidelity of Physically Based Lidar Sensor Models

Rosenberger

Holder

Zirulnik

et al. 2018

2018 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Safety validation tests of automated driving (AD) use simulated environments and perception sensor models. To achieve the level of fidelity needed for safety approval, such sensor simulations can be physically based. For formulating requirements for sensor models to be used in such test frameworks, the extent to which they must include physical effects should be determined. One approach is to clarify their relevance for following processing steps like object detection or mapping. But at first, an analysis is needed to determine, which effects are relevant and if they can be implemented at all. In this work, we focus on one lidar sensor and analyze its observable real world sensor behavior to derive the possible effects, physically based lidar sensor models can include. Consequently, we describe environmental parameters that could be considered to influence physically based lidar sensor models. By investigating the specifications given by the manufacturer with own measurements, we show that some of them should be implemented in a dynamic manner. In conclusion, we enable to formulate detailed requirements for sensor models, as their actual possible fidelity is presented.

show abstract

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Riedmaier

Danquah

Schick

et al. 2020

Arch Computat Methods Eng

View full text Add to dashboard Cite

Simulation is becoming increasingly important in the development, testing and approval process in many areas of engineering, ranging from finite element models to highly complex cyber-physical systems such as autonomous cars. Simulation must be accompanied by model verification, validation and uncertainty quantification (VV&UQ) activities to assess the inherent errors and uncertainties of each simulation model. However, the VV&UQ methods differ greatly between the application areas. In general, a major challenge is the aggregation of uncertainties from calibration and validation experiments to the actual model predictions under new, untested conditions. This is especially relevant due to high extrapolation uncertainties, if the experimental conditions differ strongly from the prediction conditions, or if the output quantities required for prediction cannot be measured during the experiments. In this paper, both the heterogeneous VV&UQ landscape and the challenge of aggregation will be addressed with a novel modular and unified framework to enable credible decision making based on simulation models. This paper contains a comprehensive survey of over 200 literature sources from many application areas and embeds them into the unified framework. In addition, this paper analyzes and compares the VV&UQ methods and the application areas in order to identify strengths and weaknesses and to derive further research directions. The framework thus combines a variety of VV&UQ methods, so that different engineering areas can benefit from new methods and combinations. Finally, this paper presents a procedure to select a suitable method from the framework for the desired application.

show abstract

Validation of vehicle environment sensor models

Cited by 28 publications

References 14 publications

Benchmarking and Functional Decomposition of Automotive Lidar Sensor Models

Benchmarking and Functional Decomposition of Automotive Lidar Sensor Models

Analysis of Real World Sensor Behavior for Rising Fidelity of Physically Based Lidar Sensor Models

Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Contact Info

Product

Resources

About