A Large-Scale Technology Evaluation Study: Effects of Model-based Analysis and Testing

Abstract

Besides model-based development, model-based quality assurance and the tighter integration of static and dynamic quality assurance activities are becoming increasingly relevant in the development of software-intensive systems. Thus, this paper reports on an empirical study aimed at investigating the promises regarding quality improvements and cost savings. The evaluation comprises data from 13 industry case studies conducted during a three-year large-scale research project in the transportation domain (automotive, avionics, rail system). During the evaluation, we identified major goals and strategies associated with (integrated) model-based analysis and testing and evaluated the improvements achieved. The aggregated results indicate an average cost reduction of between 29% and 34% for verification and validation and of between 22% and 32% for defect removal. Compared with these cost savings, improvements regarding test coverage (~8%), number of remaining defects (~13%), and time to market (~8%) appear less noticeable.

Model-driven software engineering in practice: privacy-enhanced filtering of network traffic

“…Model-driven engineering is a large area both in research and practice that is continually evaluated in many studies [9,11,12,15]. What sets this research apart from most evaluations is that we provide a full prototype model-driven solution as open-source software configured for use in a realistic scenario, facilitating immediate experimentation.…”

Testing of Autonomous Systems – Challenges and Current State‐of‐the‐Art

“…What is not, however, is automatic generation of test scenarios, i.e., model-based testing (MBT). It has been shown that MBT can significantly reduce the effort for testing (Kläs, Bauer, Söderqvist, Dereani, & Helle, 2015) and a number of the presented approaches in the previous section consider it a useful tool for testing autonomous systems. 4.…”