Avoiding useless mutants

Fernandes, Leonardo; Ribeiro, Márcio; Carvalho, Luiz; Gheyi, Rohit; Mongiovi, Melina; Santos, André L. M.; Cavalcanti, Ana; Ferrari, Fabiano Cutigi; Maldonado, José Carlos

doi:10.1145/3170492.3136053

Cited by 12 publications

(16 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, as done in previous work [106], we assume all mutants not killed as possibly not-equivalent. Previous work estimated 20% of mutants generated by PIT to be equivalent [29]: this might lead to underestimate the mutation scores computed in our work. However, since this study focuses more on using the mutation score to discern effective from non-effective tests, rather than predicting its exact value, we do not believe that this would drastically change our results.…”

Section: Threats To Validitymentioning

confidence: 70%

“…To compute the mutation score, we rely on PITEST 1 (or PIT). This choice is due to the fact that PIT was found to be the most mature publicly available mutation testing tool [28] and has been shown to limit the generation of equivalent mutants [29]. Moreover, it has been employed by most studies concerning mutation testing in the last years [4], [30], [31].…”

Section: Dependent Variablementioning

confidence: 99%

See 1 more Smart Citation

Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators

Grano

Palomba

Gall

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Test cases are crucial to help developers preventing the introduction of software faults. Unfortunately, not all the tests are properly designed or can effectively capture faults in production code. Some measures have been defined to assess test-case effectiveness: the most relevant one is the mutation score, which highlights the quality of a test by generating the so-called mutants, ie variations of the production code that make it faulty and that the test is supposed to identify. However, previous studies revealed that mutation analysis is extremely costly and hard to use in practice. The approaches proposed by researchers so far have not been able to provide practical gains in terms of mutation testing efficiency. This leaves the problem of efficiently assessing test-case effectiveness as still open. In this paper, we investigate a novel, orthogonal, and lightweight methodology to assess test-case effectiveness: in particular, we study the feasibility to exploit production and test-code-quality indicators to estimate the mutation score of a test case. We firstly select a set of 67 factors and study their relation with test-case effectiveness. Then, we devise a mutation score estimation model exploiting such factors and investigate its performance as well as its most relevant features. The key results of the study reveal that our estimation model only based on static features has 86% of both F-Measure and AUC-ROC. This means that we can estimate the test-case effectiveness, using source-code-quality indicators, with high accuracy and without executing the tests. As a consequence, we can provide a practical approach that is beyond the typical limitations of current mutation testing techniques.Abstract-Test cases are crucial to help developers preventing the introduction of software faults. Unfortunately, not all the tests are properly designed or can effectively capture faults in production code. Some measures have been defined to assess test-case effectiveness: the most relevant one is the mutation score, which highlights the quality of a test by generating the so-called mutants, i.e., variations of the production code that make it faulty and that the test is supposed to identify. However, previous studies revealed that mutation analysis is extremely costly and hard to use in practice. The approaches proposed by researchers so far have not been able to provide practical gains in terms of mutation testing efficiency. This leaves the problem of efficiently assessing test-case effectiveness as still open. In this paper, we investigate a novel, orthogonal, and lightweight methodology to assess test-case effectiveness: in particular, we study the feasibility to exploit production and test-code-quality indicators to estimate the mutation score of a test case. We firstly select a set of 67 factors and study their relation with test-case effectiveness. Then, we devise a mutation score estimation model exploiting such factors and investigate its performance as well as its most relevant features. The key results of the s...

show abstract

Section: Threats To Validitymentioning

confidence: 70%

Section: Dependent Variablementioning

confidence: 99%

Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators

Grano

Palomba

Gall

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…For instance, MuJava has 19 traditional operators which generate a large quantity of mutants and many of these mutants are redundant or duplicate of each other. Fernandes et al study showed that 34.3% of mutants generated by MuJava are duplicates and 18.6% of generated mutants are equivalents. They designed 37 transformation rules for 10 MuJava operators to eliminate useless mutants.…”

Section: Methodsmentioning

confidence: 99%

A machine learning approach for classification of equivalent mutants

Naeem

Lin

Naeem

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

Mutation testing is a fault-based technique to test the quality of test suites by inducing artificial syntactic faults or mutants in a source program. However, some mutants have the same semantics as original program and cannot be detected by any test suite input known as equivalent mutants. Equivalent mutant problem (EMP) is undecidable as it requires manual human effort to identify a mutant as equivalent or killable. The constraint-based testing (CBT) theory suggests the use of mathematical constraints which can help reveal some equivalent mutants using mutant features. In this paper, we consider three metrics of CBT theory, ie, reachability, necessity, and sufficiency to extract feature constraints from mutant programs. Constraints are extracted using program dependency graphs. Other features such as degree of significance, semantic distance, and information entropy of mutants are also extracted to build a binary classification model. Machine learning algorithms such as Random Forest, GBT, and SVM are applied under two application scenarios (split-project and cross-project) on ten Java programs to predict equivalent mutants. The analysis of the study demonstrates that that the proposed techniques not only improves the efficiency of the equivalent mutant detection but also reduces the effort required to perform it with small accuracy loss.

show abstract

“…However, some mutants are equivalent and do not contribute to the test assessment process because they have the same behavior as the original program [Budd and Angluin 1982, Jia and Harman 2011, Madeyski et al 2014. So, these mutants are useless [Fernandes et al 2017]. Listing 1 illustrates one equivalent mutant.…”

Section: Introductionmentioning

confidence: 99%

“…We encode a theory of equivalence in Z3 and use its theorem prover [de Moura and Bjørner 2008] to automatically identify equivalent mutants (Section 2). Our technique is lightweight, we do not need to create mutants, compile them, create test suites, and execute them, as previous works [Fernandes et al 2017]. Our technique automatically identifies and prove 13 equivalences for seven mutation targets.…”

Section: Introductionmentioning

confidence: 99%

A Lightweight Technique to Identify Equivalent Mutants

Souza¹,

Gheyi²

2020

Anais Estendidos Do XI Congresso Brasileiro De Software: Teoria E Prática (CBSoft 2020)

Self Cite

View full text Add to dashboard Cite

Mutation analysis is a popular but costly approach to assess the quality of test suites. Equivalent mutants are useless and contribute to increase costs. We propose a lightweight technique to identify equivalent mutants by proving equivalences with Z3 in the context of weak mutation testing. To evaluate our approach, we apply our technique for 40 mutation targets (mutations of an expression or statement) and automatically identify 13 equivalent mutations for seven mutation targets. We manually confirm that the equivalent mutants detected by our technique are indeed equivalent. Moreover, we evaluate our approach in the context of strong mutation testing against mutants generated by MuJava for 5 projects. Our technique detects all equivalent mutants detected by TCE. The results of our technique can be useful to improve mutation testing tools by avoiding the application of 13 mutations for 7 mutation targets.

show abstract

Avoiding useless mutants

Cited by 12 publications

References 22 publications

Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators

Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators

A machine learning approach for classification of equivalent mutants

A Lightweight Technique to Identify Equivalent Mutants

Contact Info

Product

Resources

About