Analyzing the validity of selective mutation with dominator mutants

Kurtz, Bob; Ammann, Paul; Offutt, Jeff; Delamaro, Márcio Eduardo; Kurtz, Mariet; Gökçe, Nida

doi:10.1145/2950290.2950322

Cited by 75 publications

(100 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We then select and prioritise our mutants. To evaluate and compare the studied approaches with respect to fault revelation, we follow a typical procedures (Titcheu Chekam et al 2017;Kurtz et al 2016;Namin et al 2008) by randomly selecting test cases, from the formed test pools, that kill the selected mutants. In case none of the available test cases on our test pool kills the mutant we treat it as equivalent.…”

Section: Methodsmentioning

confidence: 99%

“…In the literature many mutant selection methods have been proposed (Papadakis et al 2018a;Ferrari et al 2018) by restricting the considered mutants according to their types, i.e., applying one or more mutant operators. Empirical studies (Kurtz et al 2016;Deng et al 2013), have shown that the most successful strategies are the statement deletion (Deng et al 2013) and the E-Selective mutant set (Offutt et al 1993(Offutt et al , 1996a. We therefore compare our approach with these methods.…”

Section: Mutant Selectionmentioning

confidence: 99%

“…According to the study of Offutt et al (1996a) this set has the same strengths as a much larger comprehensive set of operators. Although there is empirical evidence demonstrating that the E-Selective set has lower strengths than a more comprehensive set of operators (Kurtz et al 2016), it still provides a very good trade-off beetween selected mutants and strengths (Kurtz et al 2016).…”

Section: E-selective Mutant Selectionmentioning

confidence: 99%

See 2 more Smart Citations

Selecting fault revealing mutants

et al. 2019

View full text Add to dashboard Cite

Mutant selection refers to the problem of choosing, among a large number of mutants, the (few) ones that should be used by the testers. In view of this, we investigate the problem of selecting the fault revealing mutants, i.e., the mutants that are killable and lead to test cases that uncover unknown program faults. We formulate two variants of this problem: the fault revealing mutant selection and the fault revealing mutant prioritization. We argue and show that these problems can be tackled through a set of 'static' program features and propose a machine learning approach, named FaRM, that learns to select and rank killable and fault revealing mutants. Experimental results involving 1,692 real faults show the practical benefits of our approach in both examined problems. Our results show that FaRM achieves a good trade-off between application cost and effectiveness (measured in terms of faults revealed). We also show that FaRM outperforms all the existing mutant selection methods, i.e., the random mutant sampling, the selective mutation and defect prediction (mutating the code areas pointed by defect prediction). In particular, our results show that with respect to mutant selection, our approach reveals 23% to 34% more faults than any of the baseline methods, while, with respect to mutant prioritization, it achieves higher average percentage of revealed faults with a median difference between 4% and 9% (from the random mutant orderings).

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Mutant Selectionmentioning

confidence: 99%

Section: E-selective Mutant Selectionmentioning

confidence: 99%

See 1 more Smart Citation

Selecting fault revealing mutants

et al. 2019

View full text Add to dashboard Cite

show abstract

“…However, most of the articles are not supported by empirical or theoretical studies that show a certain subset of mutation operators can represent the whole mutation operator set. As far as we know, most studies on selective mutation are merely based on Fortran [27,30,31] and C [32,193,194,195] programs. Thereby, we recommend more empirical studies on selective mutation in programming languages other than Fortran).…”

Section: Observationsmentioning

confidence: 99%

“…• R6: More empirical studies on the selective mutation method can pay attention to programming languages other than Fortran and C. As mentioned in Section 4.2.4, selective mutation is used by all the studies in our research scope. However, the selection of a subset of mutation operators in most papers is not well supported by existing empirical studies, except for Fortran [27,30,31] and C [32,193,194,195]. Selective mutation requires more empirical studies to explore whether a certain subset of mutation operators can be applied in different programming languages.…”

Section: Recommendation For Future Researchmentioning

confidence: 99%

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s, and remarkable advances have been achieved in its concepts, theory, technology, and empirical evidence.While the most influential realisations have been summarised by existing literature reviews, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 191 papers published between 1981 and 2015. In particular, we analysed in which quality assurance processes mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the inherent core problems of mutation testing, ie, the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped.Based on our observations, we made 9 recommendations for future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner. KEYWORDSapplication, mutation testing, systematic literature review 1 INTRODUCTION Mutation testing is defined by Jia and Harman [1] as a fault-based testing technique, which provides a testing criterion called the mutation adequacy score. This score can be used to measure the effectiveness of a test set in terms of its ability to detect faults [1]. The principle of mutation testing is to introduce syntactic changes into the original program to generate faulty versions (called mutants) according to well-defined rules (mutation operators) [2]. Mutation testing originated in the 1970s with works from Lipton [3], DeMillo et al. [4], and Hamlet [5] and has been a very active research field over the last few decades. The activeness of the field is in part evidenced by the extensive survey of more than 390 papers on mutation testing that Jia and Harman published in 2011 [1]. Jia and Harman's survey highlights the research achievements that have been made over the years, including the development of tools for a variety of languages and empirical studies performed [1]. Additionally, they highlight some of the actual and inherent problems of mutation testing, among others: (1) the high computational cost caused by generating and executing the numerous mutants and (2) the tremendous time-consuming human investigation required by the test oracle problem and equivalent mutant detection.While existing surveys (eg [1,2,6]) provide us with a great overview of the most influential realisations in research, we lack insight into how mutation testing is actually applied. Specifically, we are interested in analysing ...

show abstract

Analysing the combination of cost reduction techniques in Android mutation testing

Usaola

Rodríguez-Trujillo

2021

Software Testing Verif & Rel

View full text Add to dashboard Cite

When applied to mobile software, mutation testing is particularly costly due to the deployment of the app under test onto the device: if one deployment is made for each generated mutant, the execution time becomes unapproachable. This paper analyses how the combination of different cost reduction techniques improves the execution time of mutation testing in mobile apps. The techniques reviewed and combined are mutant schema, parallel execution and two different ways of executing tests against the mutants (all against all and all against mutants remaining alive), as well as greedy algorithm for reducing the test suite size. This paper also presents a mathematical model of cost reduction and checks its validity with several experiments. Furthermore, the exhaustive and long experimentation has led the authors to compile a set of good practices which are also presented in a set of lessons learned.

show abstract

Analyzing the validity of selective mutation with dominator mutants

Cited by 75 publications

References 31 publications

Selecting fault revealing mutants

Selecting fault revealing mutants

A systematic literature review of how mutation testing supports quality assurance processes

Analysing the combination of cost reduction techniques in Android mutation testing

Contact Info

Product

Resources

About