A Vehicle-in-the-Loop Methodology for Evaluating Automated Driving Functions in Virtual Traffic

Solmaz, Selim; Rudigier, Martin; Mischinger, Marlies

doi:10.1109/iv47402.2020.9304811

Cited by 24 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [166], Fayazi et al propose a ViL testing method that can configure an intersection specific environment to verify the scheduling of the ADS without physical signal lights. In [167], Solmaz et al propose a testing framework that allows real-time integration of simulated traffic scenarios in the ADS testing. In [168], Chen et al propose a ViL verification method which can reconstruct scenarios based on OpenStreetMap [156] and inject virtual perception data and real data into the perception module.…”

Section: 22mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Automated Driving System Testing: Landscapes and Trends

Tang¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automated Driving Systems (ADS) have made great achievements in recent years thanks to the efforts from both academia and industry. A typical ADS is composed of multiple modules, including sensing, perception, planning and control, which brings together the latest advances in multiple domains. Despite these achievements, safety assurance of the systems is still of great significance, since the unsafe behavior of ADS can bring catastrophic consequences and unacceptable economic and social losses. Testing is an important approach to system validation for the deployment in practice; in the context of ADS, it is extremely challenging, due to the system complexity and multidisciplinarity. There has been a great deal of literature that focuses on the testing of ADS, and a number of surveys have also emerged to summarize the technical advances. However, most of these surveys focus on the system-level testing that is performed within software simulators, and thereby ignore the distinct features of individual modules. In this paper, we provide a comprehensive survey on the existing ADS testing literature, which takes into account both module-level and system-level testing. Specifically, we make the following contributions: (1) we build a threat model that reveals the potential safety threats for each module of an ADS; (2) we survey the module-level testing techniques for ADS and highlight the technical differences affected by the properties of the modules; (3) we also survey the system-level testing techniques, but we focus on empirical studies that take a bird's-eye view on the system, the problems due to the collaborations between modules, and the gaps between ADS testing in simulators and real world; (4) we identify the challenges and opportunities in ADS testing, which facilitates the future research in this field. CCS Concepts: • Computer systems organization → Embedded systems; • Software and its engineering → Software verification and validation; • Security and privacy → Systems security; • Computing methodologies → Artificial intelligence.

show abstract

Section: 22mentioning

confidence: 99%

“…CarMaker [214] Platform for model-based design and testing of vehicle dynamics [167,176] In conclusion, programming languages are increasingly deemed as powerful weapons for test case generation in ADS testing, but they are not widely adopted yet in practice.…”

Section: 3g Videos -2020mentioning

confidence: 99%

A Survey on Automated Driving System Testing: Landscapes and Trends

Tang¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This proposes an impossible task since the demonstration of the vehicle performance needs to be completed prior to the release for consumer use. Hence, reducing the development effort for ADAS functions and eventually enabling AD functions demand the extension of conventional test methods, e.g., physical test drives, with simulations in virtual test environments [4,12], or mixed methods combining the both testing abstraction levels [13][14][15][16].…”

Section: Virtual Testing Of Adas/ad Functionsmentioning

confidence: 99%

Development and Experimental Validation of an Intelligent Camera Model for Automated Driving

Genser

Muckenhuber

Solmaz

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

The virtual testing and validation of advanced driver assistance system and automated driving (ADAS/AD) functions require efficient and realistic perception sensor models. In particular, the limitations and measurement errors of real perception sensors need to be simulated realistically in order to generate useful sensor data for the ADAS/AD function under test. In this paper, a novel sensor modeling approach for automotive perception sensors is introduced. The novel approach combines kernel density estimation with regression modeling and puts the main focus on the position measurement errors. The modeling approach is designed for any automotive perception sensor that provides position estimations at the object level. To demonstrate and evaluate the new approach, a common state-of-the-art automotive camera (Mobileye 630) was considered. Both sensor measurements (Mobileye position estimations) and ground-truth data (DGPS positions of all attending vehicles) were collected during a large measurement campaign on a Hungarian highway to support the development and experimental validation of the new approach. The quality of the model was tested and compared to reference measurements, leading to a pointwise position error of 9.60% in the lateral and 1.57% in the longitudinal direction. Additionally, the modeling of the natural scattering of the sensor model output was satisfying. In particular, the deviations of the position measurements were well modeled with this approach.

show abstract

“…Highly automated and soon fully autonomous vehicles will revolutionize road transportation [1,2]. Although this trend is very spectacular, in the background a vast amount of engineering research and development (R&D) work is required, i.e., Connected Automated Vehicles (CAVs) and all related technologies require complex testing and validation procedures [3][4][5][6].…”

Section: Introductionmentioning

confidence: 99%

“…When using the terminology of a recent V2X testing survey [11], our paper covers V2X performance testing (especially latency testing) that involves vehicle gateway testing. Nevertheless, the major achievement here is presenting how live field-testing has been planned and executed in order to evoke further, digital-twin-based validations through Vehicle-in-the-Loop (ViL) [5,12] or Scenario-in-the-Loop (SciL) [13] methodologies. The concepts and perspectives of ViL and SciL have been summarized well by [14].…”

Section: Introductionmentioning

confidence: 99%

Motorway Measurement Campaign to Support R&D Activities in the Field of Automated Driving Technologies

Tihanyi

Tettamanti

Csonthó

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

A spectacular measurement campaign was carried out on a real-world motorway stretch of Hungary with the participation of international industrial and academic partners. The measurement resulted in vehicle based and infrastructure based sensor data that will be extremely useful for future automotive R&D activities due to the available ground truth for static and dynamic content. The aim of the measurement campaign was twofold. On the one hand, road geometry was mapped with high precision in order to build Ultra High Definition (UHD) map of the test road. On the other hand, the vehicles—equipped with differential Global Navigation Satellite Systems (GNSS) for ground truth localization—carried out special test scenarios while collecting detailed data using different sensors. All of the test runs were recorded by both vehicles and infrastructure. The paper also showcases application examples to demonstrate the viability of the collected data having access to the ground truth labeling. This data set may support a large variety of solutions, for the test and validation of different kinds of approaches and techniques. As a complementary task, the available 5G network was monitored and tested under different radio conditions to investigate the latency results for different measurement scenarios. A part of the measured data has been shared openly, such that interested automotive and academic parties may use it for their own purposes.

show abstract

A Vehicle-in-the-Loop Methodology for Evaluating Automated Driving Functions in Virtual Traffic

Cited by 24 publications

References 9 publications

A Survey on Automated Driving System Testing: Landscapes and Trends

A Survey on Automated Driving System Testing: Landscapes and Trends

Development and Experimental Validation of an Intelligent Camera Model for Automated Driving

Motorway Measurement Campaign to Support R&D Activities in the Field of Automated Driving Technologies

Contact Info

Product

Resources

About