A Systematic Review of Cost Reduction Techniques for Mutation Testing: Preliminary Results

Ferrari, Fabiano Cutigi; Pizzoleto, Alessandro Viola; Offutt, Jeff

doi:10.1109/icstw.2018.00021

Cited by 8 publications

(6 citation statements)

References 127 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The fault revealing mutant selection goal is different from that of the "traditional" mutant reduction techniques, which is to reduce the number of mutants (Offutt et al 1996a;Wong and Mathur 1995b;Ferrari et al 2018;Papadakis et al 2018a). Mutant reduction strategies focus on selecting a small set of mutants that is representative of the larger set.…”

Section: Problem Definitionmentioning

confidence: 99%

“…In the literature many mutant selection methods have been proposed (Papadakis et al 2018a;Ferrari et al 2018) by restricting the considered mutants according to their types, i.e., applying one or more mutant operators. Empirical studies (Kurtz et al 2016;Deng et al 2013), have shown that the most successful strategies are the statement deletion (Deng et al 2013) and the E-Selective mutant set (Offutt et al 1993(Offutt et al , 1996a.…”

Section: Mutant Selectionmentioning

confidence: 99%

See 1 more Smart Citation

Selecting fault revealing mutants

et al. 2019

View full text Add to dashboard Cite

Mutant selection refers to the problem of choosing, among a large number of mutants, the (few) ones that should be used by the testers. In view of this, we investigate the problem of selecting the fault revealing mutants, i.e., the mutants that are killable and lead to test cases that uncover unknown program faults. We formulate two variants of this problem: the fault revealing mutant selection and the fault revealing mutant prioritization. We argue and show that these problems can be tackled through a set of 'static' program features and propose a machine learning approach, named FaRM, that learns to select and rank killable and fault revealing mutants. Experimental results involving 1,692 real faults show the practical benefits of our approach in both examined problems. Our results show that FaRM achieves a good trade-off between application cost and effectiveness (measured in terms of faults revealed). We also show that FaRM outperforms all the existing mutant selection methods, i.e., the random mutant sampling, the selective mutation and defect prediction (mutating the code areas pointed by defect prediction). In particular, our results show that with respect to mutant selection, our approach reveals 23% to 34% more faults than any of the baseline methods, while, with respect to mutant prioritization, it achieves higher average percentage of revealed faults with a median difference between 4% and 9% (from the random mutant orderings).

show abstract

Section: Problem Definitionmentioning

confidence: 99%

Section: Mutant Selectionmentioning

confidence: 99%

Selecting fault revealing mutants

et al. 2019

View full text Add to dashboard Cite

show abstract

“…b We are now executing the mutant index with value 3 that is the third mutant proceeding from b + c. This value has been saved in the aforementioned file and is assigned to currentMutant in the loadCurrentMutant method. This value (3) is searched in the array passed in the variable parameters set (it was in previous studies [1,2,24,44]) and found with location = 2. Then, the method returns a/b.…”

Section: Mutant Schema Generationmentioning

confidence: 99%

“…The effectiveness of mutation testing has been demonstrated in many empirical studies [1] although it has the important drawback of its high computational cost, which is closely related to the number of mutants generated. Thus, cost reduction in mutation testing is a very active research matter [2][3]. Most of these studies are focused on the reduction of the number of mutants and on how tests are executed against mutants.…”

Section: Introductionmentioning

confidence: 99%

Analysing the combination of cost reduction techniques in Android mutation testing

Usaola

Rodríguez-Trujillo

2021

Software Testing Verif & Rel

View full text Add to dashboard Cite

When applied to mobile software, mutation testing is particularly costly due to the deployment of the app under test onto the device: if one deployment is made for each generated mutant, the execution time becomes unapproachable. This paper analyses how the combination of different cost reduction techniques improves the execution time of mutation testing in mobile apps. The techniques reviewed and combined are mutant schema, parallel execution and two different ways of executing tests against the mutants (all against all and all against mutants remaining alive), as well as greedy algorithm for reducing the test suite size. This paper also presents a mathematical model of cost reduction and checks its validity with several experiments. Furthermore, the exhaustive and long experimentation has led the authors to compile a set of good practices which are also presented in a set of lessons learned.

show abstract

“…However, identifying equivalent mutants is a costly task that places a significant burden on the tester. Despite being an undecidable problem, many researchers have proposed different approaches in the past to reduce its impact [6]. A recently proposed and promising strategy, especially because of its cost-benefit trade-off, leverages compiler optimizations to detect trivial equivalent mutants [19].…”

mentioning

confidence: 99%

Study of trivial compiler equivalence on C++ object-oriented mutation operators

Delgado‐Pérez

Segura

2019

Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing

View full text Add to dashboard Cite

Trivial Compiler Equivalence (TCE) has been recently proposed as an effective technique to detect equivalences between programs, where two or more programs are equivalent if the compiler produces the same binary code. Mutation testing can greatly benefit from TCE as a way to reveal some equivalent and duplicate mutants, which traditionally hinder the applicability of the technique. For instance, previous research has shown that about 28% of the mutants generated by traditional mutation operators in C programs can be removed using TCE. However, the effectiveness of TCE has not been assessed with class-level operators, where the percentage of equivalent mutants is known to be higher than when using traditional ones. In this paper, we present an empirical study on the effectiveness of TCE at identifying equivalent and duplicate mutants using C++ class operators. The results show that TCE is helpful to discard equivalent and duplicate mutants: 241 out of 1,987 (12%) in our study, including 189 out of 684 (27.6%) manually-identified equivalent mutants. Large differences were observed among the different case studies, especially in the detection rate of equivalent mutants, which ranged from 4% to 45%.

show abstract

A Systematic Review of Cost Reduction Techniques for Mutation Testing: Preliminary Results

Cited by 8 publications

References 127 publications

Selecting fault revealing mutants

Selecting fault revealing mutants

Analysing the combination of cost reduction techniques in Android mutation testing

Study of trivial compiler equivalence on C++ object-oriented mutation operators

Contact Info

Product

Resources

About