Experimental Evaluation of SDL and One-Op Mutation for C

Delamaro, Márcio Eduardo; Deng, Lin; Durelli, Vinícius H. S.; Li, Nan; Offutt, Jeff

doi:10.1109/icst.2014.33

Cited by 44 publications

(35 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although finding a minimum test set is, like many optimization problems, computationally hard 3 , generating a minimal test set is straightforward. Algorithm 1 generates a minimal test set with time complexity |T | * |M |.…”

Section: Definition 1: Minimal Test Setsmentioning

confidence: 99%

See 1 more Smart Citation

Establishing Theoretical Minimal Sets of Mutants

Ammann

Delamaro

Offutt

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

Self Cite

147

142

View full text Add to dashboard Cite

Abstract-Mutation analysis generates tests that distinguish variations, or mutants, of an artifact from the original. Mutation analysis is widely considered to be a powerful approach to testing, and hence is often used to evaluate other test criteria in terms of mutation score, which is the fraction of mutants that are killed by a test set. But mutation analysis is also known to provide large numbers of redundant mutants, and these mutants can inflate the mutation score. While mutation approaches broadly characterized as reduced mutation try to eliminate redundant mutants, the literature lacks a theoretical result that articulates just how many mutants are needed in any given situation. Hence, there is, at present, no way to characterize the contribution of, for example, a particular approach to reduced mutation with respect to any theoretical minimal set of mutants. This paper's contribution is to provide such a theoretical foundation for mutant set minimization. The central theoretical result of the paper shows how to minimize efficiently mutant sets with respect to a set of test cases. We evaluate our method with a widely-used benchmark.

show abstract

Section: Definition 1: Minimal Test Setsmentioning

confidence: 99%

“…Since m 1 and m 2 are distinguished and redundant, both can safely be removed from M without altering the resulting minimal test sets, thereby yielding a minimal mutant set of {m 3 , m 4 }. In this example, there is only one minimal mutant set.…”

Section: Minimal Sets Of Mutantsmentioning

confidence: 99%

Establishing Theoretical Minimal Sets of Mutants

Ammann

Delamaro

Offutt

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

Self Cite

147

142

View full text Add to dashboard Cite

show abstract

“…These mutants were generated by a single mutation operator, Statement Deletion (SSDL). We chose SSDL for two reasons: (1) it is universally applicable to all programs; and (2) it has been studied in recent papers as an alternative to using the complete set of mutation operators [1,7,8].…”

Section: Methodsmentioning

confidence: 99%

“…Some test cases were provided with the Siemens programs 1 and we generated additional tests by hand to kill all nonequivalent mutants. For the remaining programs, we also generated tests by hand to kill all non-equivalent mutants.…”

Section: Methodsmentioning

confidence: 99%

“…In a recent paper [1] the authors faced the problem of selecting adequate test sets to evaluate the cost and effectiveness of individual mutation operators. We used the approach pioneered by Frankl [2] of first generating a "universe" of test cases that were adequate for the complete set of mutants, and then selected subsets that were adequate for the specific mutants being studied.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Assessing the Influence of Multiple Test Case Selection on Mutation Experiments

Delamaro¹,

Offutt

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops

Self Cite

View full text Add to dashboard Cite

Abstract-Mutation testing is widely used in experiments. Some papers experiment with mutation directly, while others use it to introduce faults to measure the effectiveness of tests created by other methods. There is some random variation in the mutation score depending on the specific test values used. When generating tests to use in experiments, a common, although not universal practice, is to generate multiple sets of tests to satisfy the same criterion or according to the same procedure, and then to compute their average performance. Averaging over multiple test sets is thought to reduce the variation in the mutation score. This practice is extremely expensive when tests are generated by hand (as is common) and as the number of programs increase (a current positive trend in software engineering experimentation).The research reported in this short paper asks a simple and direct question: do we need to generate multiple sets of test cases? That is, how do different test sets influence the cost and effectiveness results? In a controlled experiment, we generated 10 different test cases to be adequate for the Statement Deletion (SSDL) mutation operator for 39 small programs and functions, and then evaluated how they differ in terms of cost and effectiveness. We found that averaging over multiple programs was effective in reducing the variance in the mutation scores introduced by specific tests.

show abstract

Learning‐based mutant reduction using fine‐grained mutation operators

Kim

Hong

2021

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryFor mutation testing, the huge cost of running test suites on a large number of mutants has been a serious obstacle. To resolve this problem, we propose a learning‐based mutant reduction technique MuTrain. MuTrain uses cost‐considerate linear regression (i.e., CLARS) to learn a mutation model, which predicts the mutation score of a test suite based on the mutation testing results of a previous version of a target program. Then, MuTrain applies the mutation model for subsequent versions to predict mutation scores with significantly fewer mutants. For effective mutant reduction and accurate mutation score prediction, MuTrain uses fine‐grained mutation operators refined from the existing coarse‐grained mutation operators. The experiment results show that MuTrain reduces the number of mutants effectively (i.e., selecting only 1.6% of mutants). Moreover, MuTrain predicts mutation score far more accurately than the existing mutant reduction techniques and random mutant selection. We also found that MuTrain achieves much greater mutant reduction when it uses the fine‐grained mutation operators than the traditional coarse‐grained mutation operators (i.e., 1.6% vs. 14.6%).

show abstract

Experimental Evaluation of SDL and One-Op Mutation for C

Cited by 44 publications

References 29 publications

Establishing Theoretical Minimal Sets of Mutants

Establishing Theoretical Minimal Sets of Mutants

Assessing the Influence of Multiple Test Case Selection on Mutation Experiments

Learning‐based mutant reduction using fine‐grained mutation operators

Contact Info

Product

Resources

About