Unified Framework and Survey for Model Verification, Validation and Uncertainty Quantification

Abstract

Simulation is becoming increasingly important in the development, testing and approval process in many areas of engineering, ranging from finite element models to highly complex cyber-physical systems such as autonomous cars. Simulation must be accompanied by model verification, validation and uncertainty quantification (VV&UQ) activities to assess the inherent errors and uncertainties of each simulation model. However, the VV&UQ methods differ greatly between the application areas. In general, a major challenge is the aggregation of uncertainties from calibration and validation experiments to the actual model predictions under new, untested conditions. This is especially relevant due to high extrapolation uncertainties, if the experimental conditions differ strongly from the prediction conditions, or if the output quantities required for prediction cannot be measured during the experiments. In this paper, both the heterogeneous VV&UQ landscape and the challenge of aggregation will be addressed with a novel modular and unified framework to enable credible decision making based on simulation models. This paper contains a comprehensive survey of over 200 literature sources from many application areas and embeds them into the unified framework. In addition, this paper analyzes and compares the VV&UQ methods and the application areas in order to identify strengths and weaknesses and to derive further research directions. The framework thus combines a variety of VV&UQ methods, so that different engineering areas can benefit from new methods and combinations. Finally, this paper presents a procedure to select a suitable method from the framework for the desired application.

Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles

“…Since models have a long history and are used in numerous application areas, there is a myriad of literature on model validation. We gave a comprehensive survey about model Verification, Validation and Uncertainty Quantification (VV&UQ) approaches in [20]. We developed a generic modular framework and embedded most of the literature approaches and application areas within it.…”

“…At this point, we highlight individual aspects that are integral to this paper's central theme and understanding. The interested reader is referred to [20] regarding a more detailed introduction and theory. In the later sections, we will present this framework in a specific manifestation for the virtual-based safeguarding of AVs.…”

“…Our process is illustrated in Figure 2 for the LKFT use case. It builds on the previous use case independent work in [20]. It distinguishes the validation from the application domain.…”

“…The scenario parameters can also be combined to the input vector x = 1 v x a y,re f ∈ R N x +1 to get a compact notation for the remaining sections and to ensure consistency with our previous paper [20]. Each coverage-based validation scenario x v yields for both the simulation model g m and the physical system g s a minimum distance to line y v m and y v s , respectively.…”

Continuous Development and Management of Credible Modelica Models

“…Since the scope of model calibration, validation and verification is very large, we can only give here a short overview in the context of a typical use case for a multi-physical Modelica model. For a broader overview on the topic, a recent paper (Riedmaier et al 2021) goes into more details and gives many additional references.…”