Towards automating the generation of mutation tests

Papadakis, Mike; Malevris, Nicos; Kallia, Maria

doi:10.1145/1808266.1808283

Cited by 34 publications

(43 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Fraser and Zeller [60] used search-based software testing (SBST) to generate tests for mutant killing. Zhang et al [16] and Papadakis et al [17] used dynamic symbolic execution (DSE) to generate tests for mutant killing. Harman et al [14] combined SBST and DSE techniques to generate tests that kill multiple mutants at a time.…”

Section: B Applications Of Mutation Testingmentioning

confidence: 99%

Operator-based and random mutant selection: Better together

Zhang

Gligoric

Marinov

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

Abstract-Mutation testing is a powerful methodology for evaluating the quality of a test suite. However, the methodology is also very costly, as the test suite may have to be executed for each mutant. Selective mutation testing is a well-studied technique to reduce this cost by selecting a subset of all mutants, which would otherwise have to be considered in their entirety. Two common approaches are operator-based mutant selection, which only generates mutants using a subset of mutation operators, and random mutant selection, which selects a subset of mutants generated using all mutation operators. While each of the two approaches provides some reduction in the number of mutants to execute, applying either of the two to medium-sized, realworld programs can still generate a huge number of mutants, which makes their execution too expensive. This paper presents eight random sampling strategies defined on top of operatorbased mutant selection, and empirically validates that operatorbased selection and random selection can be applied in tandem to further reduce the cost of mutation testing. The experimental results show that even sampling only 5% of mutants generated by operator-based selection can still provide precise mutation testing results, while reducing the average mutation testing time to 6.54% (i.e., on average less than 5 minutes for this study).

show abstract

Section: B Applications Of Mutation Testingmentioning

confidence: 99%

Operator-based and random mutant selection: Better together

Zhang

Gligoric

Marinov

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

show abstract

“…Except for evaluating test power, mutation testing has also been shown to be suitable for simulating real faults in software testing experimentation [9], [10], [11], localizing faults [12], [13], [14], performing model transformations [15], and guiding test generation [16], [17], [18], [19], [20], [21].…”

Section: Introductionmentioning

confidence: 99%

Predictive Mutation Testing

Zhang

Harman

et al. 2019

IIEEE Trans. Software Eng.

116

110

View full text Add to dashboard Cite

Abstract-Test suites play a key role in ensuring software quality. A good test suite may detect more faults than a poor-quality one. Mutation testing is a powerful methodology for evaluating the fault-detection ability of test suites. In mutation testing, a large number of mutants may be generated and need to be executed against the test suite under evaluation to check how many mutants the test suite is able to detect, as well as the kind of mutants that the current test suite fails to detect. Consequently, although highly effective, mutation testing is widely recognized to be also computationally expensive, inhibiting wider uptake. To alleviate this efficiency concern, we propose Predictive Mutation Testing (PMT): the first approach to predicting mutation testing results without executing mutants. In particular, PMT constructs a classification model, based on a series of features related to mutants and tests, and uses the model to predict whether a mutant would be killed or remain alive without executing it. PMT has been evaluated on 163 real-world projects under two application scenarios (cross-version and cross-project). The experimental results demonstrate that PMT improves the efficiency of mutation testing by up to 151.4X while incurring only a small accuracy loss. It achieves above 0.80 AUC values for the majority of projects, indicating a good tradeoff between the efficiency and effectiveness of predictive mutation testing. Also, PMT is shown to perform well on different tools and tests, be robust in the presence of imbalanced data, and have high predictability (over 60% confidence) when predicting the execution results of the majority of mutants.

show abstract

“…DSE has proved to be an effective means of satisfying both the reachability and infection conditions [12,39] and, as a result, there has been work on DSE as a technique for achieving weak mutation adequacy [34,37,42]. However, it has not been adapted to handle strong mutation.…”

Section: Weakly Killing Mutantsmentioning

confidence: 99%

“…We use a different variant of the DSE algorithm to that previously used for mutation testing [34,37,42]. Our reachability approach is the same as previous work and this is inherited from the standard DSE approach to branch coverage [12,39].…”

Section: Handling Higher Order Mutantsmentioning

confidence: 99%

“…Previous work on the generation of test data to kill mutants has used traditional structural-oriented test data generation techniques, for example, traditional symbolic execution [8,26,30,31,32], Dynamic Symbolic Execution (DSE) [34,37,42] and Search Based Software Testing (SBST) [4,11]. However, all of the existing techniques are designed to achieve only weak mutation adequacy and only for first order mutants.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Strong higher order mutation-based test data generation

Harman

Jia

Langdon

2011

Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering

119

100

View full text Add to dashboard Cite

This paper introduces SHOM, a mutation-based test data generation approach that combines Dynamic Symbolic Execution and Search Based Software Testing. SHOM targets strong mutation adequacy and is capable of killing both first and higher order mutants. We report the results of an empirical study using 17 programs, including production industrial code from ABB and Daimler and open source code as well as previously studied subjects. SHOM achieved higher strong mutation adequacy than two recent mutation-based test data generation approaches, killing between 8% and 38% of those mutants left unkilled by the best performing previous approach.

show abstract

Towards automating the generation of mutation tests

Cited by 34 publications

References 35 publications

Operator-based and random mutant selection: Better together

Operator-based and random mutant selection: Better together

Predictive Mutation Testing

Strong higher order mutation-based test data generation

Contact Info

Product

Resources

About