Verification validation and accreditation of simulation models

Balcı, Osman

doi:10.1145/268437.268462

Cited by 155 publications

(115 citation statements)

References 2 publications

Supporting

Mentioning

110

Contrasting

Unclassified

Order By: Relevance

“…Researchers concerned with the validation and verification of simulation software often acknowledge the need for software testing, but typically do not present techniques beyond using some sort of formal specification [30], or approaches that are common for all types of software, such as test-driven development or code reviews [4,19,27]. Others have presented techniques for exploring the simulation's potentially large input space for software testing purposes [28], but we are not aware of any work that has attempted to reveal coding defects in simulators by using a systematic approach.…”

Section: Related Workmentioning

confidence: 99%

On effective testing of health care simulation software

Murphy

Raunak

King

et al. 2011

Proceedings of the 3rd Workshop on Software Engineering in Health Care

View full text Add to dashboard Cite

Health care professionals rely on software to simulate anatomical and physiological elements of the human body for purposes of training, prototyping, and decision making. Software can also be used to simulate medical processes and protocols to measure cost effectiveness and resource utilization. Whereas much of the software engineering research into simulation software focuses on validation (determining that the simulation accurately models real-world activity), to date there has been little investigation into the testing of simulation software itself, that is, the ability to effectively search for errors in the implementation. This is particularly challenging because often there is no test oracle to indicate whether the results of the simulation are correct. In this paper, we present an approach to systematically testing simulation software in the absence of test oracles, and evaluate the effectiveness of the technique.

show abstract

Section: Related Workmentioning

confidence: 99%

On effective testing of health care simulation software

Murphy

Raunak

King

et al. 2011

Proceedings of the 3rd Workshop on Software Engineering in Health Care

View full text Add to dashboard Cite

show abstract

“…Classical, i.e. non component-based, validation methods include various stages and processes, and often result in a costly, time-consuming process [1,12]. The validation cost is further aggravated in component-based modeling and simulation where the complexity of the models increases rapidly with size, and semantic composability validation remains a non-trivial problem [4,10,25,32].…”

Section: Introductionmentioning

confidence: 99%

“…Firstly, composition is not a closed operation with respect to validation since valid components do not necessary form valid compositions [1]. Secondly, reused components are developed for different purposes and when composed may result in emergent properties [15].…”

Section: Introductionmentioning

confidence: 99%

“…model behavior aspects such as deadlock, safety, and liveness, temporal aspects, and formal measures of the validity of compositions, also called "figures of merit" [17]. Lastly, the composed model must be similar or close to the real system it abstracts [1]. Very often, the implementation of an automated process to evaluate this similarity is difficult, if not impossible [1,10,32].…”

Section: Introductionmentioning

confidence: 99%

“…Lastly, the composed model must be similar or close to the real system it abstracts [1]. Very often, the implementation of an automated process to evaluate this similarity is difficult, if not impossible [1,10,32].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An analysis of the cost of validating semantic composability

Szabo

Teo

2012

Journal of Simulation

View full text Add to dashboard Cite

Validation of semantic composability is a non-trivial problem and a key step in component-based modeling and simulation. Recent work in semantic composability validation promise to reduce verification, validation, and accreditation efforts. However, the underlying cost of current validation approaches can undermine the promised benefits, and the trade-off between validation accuracy and validation cost is not well understood. In this paper we present, to the best of our knowledge, the first quantitative study on the cost of validating semantic composability. Firstly, validation approaches are categorized into techniques that validate general model properties, and that compare the composed model execution with a reference model. Secondly, we focus on two key factors that influence validation cost: characteristics of the simulation problem and the validation approach adopted. Our study covers four representative validation approaches, two DEVS-based approaches, the Petty and Weisel formal validation, and deny-validity, and for simplicity, we use computation time as a measure of validation cost. Based on a queueing model with 1,000 components, the cost ratio between validating general model properties and model execution is 55:45. A 10% increase in the complexity of the composition structure, such as fork and join component interconnections, increases validation cost by more than half. In model execution validation, there is a trade-off between validation accuracy and validation cost, with the time-based deny-validity approach costing seven times that of timeless Petty and Weisel formalism.

show abstract

References

2019

Geographical Modeling

View full text Add to dashboard Cite

Verification validation and accreditation of simulation models

Cited by 155 publications

References 2 publications

On effective testing of health care simulation software

On effective testing of health care simulation software

An analysis of the cost of validating semantic composability

References

Contact Info

Product

Resources

About