Probability-Based Semantic Interpretation of Mutants

Abstract

Abstract-Mutation analysis is a stringent and powerful technique for evaluating the ability of a test suite to find faults. It generates a large number of mutants and applies the test suite to them one at a time. As mutation analysis is computationally expensive, it is usually performed on a subset of mutants. The competent programmer hypothesis suggests that experienced software developers are more likely to make small mistakes. It is prudent therefore to focus on semantically small mutants that represent mistakes developers are likely to make. We previously introduced a technique to assess mutant semantics using static analysis by comparing the numerical range of their symbolic output expressions. This paper extends our previous work by considering the probability the output of a mutant is the same as the original program. We show how probability-based semantic interpretation can be used to select mutants that are semantically more similar than those selected by our previous technique. In addition to numerical outputs, it also provides support for modelling the semantics of Boolean variables, strings and composite objects.

Mutation Testing Advances: An Analysis and Survey

“…To determine the effect of the mutants on the program output, Pattrick et al suggests calculating the range of values (on the numerical output expressions) that differ when mutants are killed. This method was latter refined by Pattrick et al [135] by considering the semantics (in addition to numeric ones) of Boolean variables, strings and composite objects.…”

Detecting Trivial Mutant Equivalences via Compiler Optimisations

“…To reduce this threat we carefully checked their results. However, we consider this as a minor threat since all the used tools have been used by several authors in recent studies, e.g., [6], [57], [70], [71], [72], independently of us. Furthermore, we utilised three equivalent mutant benchmark sets which were entirely built by hand.…”