Mutation Testing Approach to Evaluation of Design Models

Abstract

A well-formed design model, reflecting accurately all needs of stakeholders, contributes to a development and manufacturing of a high quality system. It is therefore of primary importance to evaluate a model to determine the degree to which the model is an accurate representation of these needs. A mutation testing based approach to evaluation and measuring of a design model with regard to its accuracy is presented in this paper. The approach focused mainly on detecting weaknesses of a model that could cause the future system to process incorrect data or crash when some unexpected situation occurs.

Mutation Testing Approach to Negative Testing

“…To the author's best knowledge, so far, only a few other researchers have followed the idea. The approach described in [27] is the closest to the approach outlined in [7] and the one further presented in this paper. The approach in [27] deals with measuring a specification-implementation concordance, but it also aims at modifying test cases directly.…”

“…A mutation operator is a transformation rule that defines how to modify certain features of the artifact undergoing mutations [5]. There is a subset of the so-called traditional mutation operators that are fairly universal and can be easily adapted for different programming or modeling languages, but in general mutation operators are formalism-specific [7,14,16,19,[32][33][34]. Therefore, application of mutation testing in different context, as in this approach, should always include defining an adequate suite of mutation operators.…”

“…Test cases obtained in this way can reflect a wide variety of unexpected usage scenarios for a system and thus contribute to better testing. The concept of mutating test cases rather than a program was outlined in [7] as a support for model evaluation. This paper builds on this concept, but it formalizes it and focuses on 2 Journal of Engineering object-oriented software systems.…”

Mutation Operators for UML Class Diagrams

“…However, some similarities can be found in Strug [18,26] Dinh-Trong et al [17] and Derezinska [16]. In the former [26], the author introduces nine mutation operators to apply manual mutations to the test suite provided for a UML/OCL-based design model instead of modifying the model, which is a different approach to that used in the present paper. In the latter [18], the author presents a classification of 16 mutation operators defined for constraints specified in OCL and used in UML/OCL-based design models.…”

Applying Mutation Testing for Assessing Test Suites Quality at Model Level

“…However, application of mutation testing at model level seems to gain popularity [1], [2], [16], [18], [19], [23], [24]. Authors of the approaches have focused mainly on selecting tests, but some of them have addressed also the problem of assessing tests at the model level [24], [23] and discussed selected aspects related to the problem of assessing tests quality at different levels [23].…”