Quality Assessment, Verification, and Validation of Modeling and Simulation Applications

Abstract

Many different types of modeling and simulation (M&S) applications are used in dozens of disciplines under diverse objectives including acquisition, analysis, education, entertainment, research, and training. M&S application verification and validation (V&V) are conducted to assess mainly the accuracy, which is one of many indicators affecting the M&S application quality. Much higher confidence can be achieved in accuracy if a quality-centered approach is used. This paper presents a quality model for assessing the quality of large-scale complex M&S applications as integrated with V&V. The guidelines provided herein should be useful for assessing the overall quality of an M&S application.

Best practices for modeling, simulation and analysis (MS&A) for homeland security applications

“…The intended use documentation may at time be variously referred to as system specification, user requirements, or functional requirements. Balci (2004) recommends use of a tool like Evaluation Environment for collaborative application of a quality model and emphasizes the need for rigorous quality assessment of M&S products including M&S requirement specification.…”

“…Over 100 V&V techniques exist (Balci 2004 The standard for modeling and simulation from National Aeronautics and Space Administration (NASA 2008) has the overall goal "to ensure that the credibility of the results from M&S is properly conveyed to those making critical decisions." Thus, while it covers a number of aspects related to use of modeling and simulation, one of the primary focus is on the credibility of the M&S results and hence on V&V. The standard defines three requirements each for verification and validation calling for documentation of the processes used and the results.…”

A modified ANP and its application in simulation credibility evaluation

“…The importance of computer simulations in the design and performance assessment of engineering systems has increased dramatically during the last three or four decades [20,21]. Verification and validation (V&V) [22] are the primary processes for quantifying and building credibility in numerical models.…”

Report generation for simulation traces with Traviando

“…A valid and verified model is expected to provide reasonably accurate values for measures of interest that are consistent with what would be observed for the system of interest. V&V is a classic topic in simulation and the Winter Simulation Conference devotes a tutorial paper to it in each year, often by Balci [3] and Sargent [9]. In particular for model verification, Sargent [9] names structured walkthroughs, correctness proofs and examining the structure properties of a simulation program for static "testing" techniques.…”