Empirical Evaluation of the Statement Deletion Mutation Operator

Deng, Lin; Offutt, Jeff; Li, Nan

doi:10.1109/icst.2013.20

Cited by 61 publications

(56 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SDL-mutation [15] is a one-op mutation version that uses Statement Deletion (SDL). Results showed that SDLmutation yields huge savings in the cost of mutation testing without a significant loss of effectiveness [14].…”

Section: Deletion Mutation Operatorsmentioning

confidence: 99%

“…The usual way to assess an operator's effectiveness is to first design a test set that kills all of its mutants, and then find how many mutants that test set kills of the complete set of mutants. In previous experiments, SDL achieved 92% effectiveness for Java [14]. Second, the cost of using SDL is low.…”

Section: Deletion Mutation Operatorsmentioning

confidence: 99%

“…A recent study [14] focused on a single operator with high effectiveness. The study found that the Statement Deletion (SDL) operator has characteristics that make it a likely choice for a single operator mutation criterion.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Designing Deletion Mutation Operators

Delamaro

Offutt

Ammann

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

Abstract-As a test criterion, mutation analysis is known for yielding very effective tests. It is also known for creating many test requirements, each of which is represented by a "mutant" that must be "killed." In recent years, researchers have found that these test requirements have a lot of duplication, in that many test requirements yield the same tests. Put another way, hundreds of mutants can usually be killed by only a few dozen tests. If we could reduce this duplication without reducing mutation's effectiveness, mutation testing could become more cost-effective. One avenue of this research has been to use only one type of mutant, the statement deletion mutation operator. Researchers have found that statement deletion mutation has relatively few mutants, but yields tests that are almost as effective as using all mutants, with the significant benefit that fewer equivalent mutants are generated. This paper extends this idea by asking a simple question: if deleting statements is a cost-effective way to design tests, will deleting other program elements also be effective? This paper presents results from mutation operators that delete variables, operators, and constants, finding that indeed, this is an efficient and effective approach.

show abstract

Section: Deletion Mutation Operatorsmentioning

confidence: 99%

Section: Deletion Mutation Operatorsmentioning

confidence: 99%

See 1 more Smart Citation

Designing Deletion Mutation Operators

Delamaro

Offutt

Ammann

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

show abstract

“…By doing this transformation, OAAN and OBBN apply in those cases.  We included SSDL because recent studies have pointed to the usefulness of this operator [41]. Moreover, the comparable study by Baker and Habli [32] analyses this operator, so it is interesting to observe if mutants from this operator are also effective in detecting deficiencies in our branch-adequate test suites.…”

Section: G Cost Reduction Techniquesmentioning

confidence: 99%

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

“…These mutants were generated by a single mutation operator, Statement Deletion (SSDL). We chose SSDL for two reasons: (1) it is universally applicable to all programs; and (2) it has been studied in recent papers as an alternative to using the complete set of mutation operators [1,7,8].…”

Section: Methodsmentioning

confidence: 99%

Assessing the Influence of Multiple Test Case Selection on Mutation Experiments

Delamaro¹,

Offutt

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops

Self Cite

View full text Add to dashboard Cite

Abstract-Mutation testing is widely used in experiments. Some papers experiment with mutation directly, while others use it to introduce faults to measure the effectiveness of tests created by other methods. There is some random variation in the mutation score depending on the specific test values used. When generating tests to use in experiments, a common, although not universal practice, is to generate multiple sets of tests to satisfy the same criterion or according to the same procedure, and then to compute their average performance. Averaging over multiple test sets is thought to reduce the variation in the mutation score. This practice is extremely expensive when tests are generated by hand (as is common) and as the number of programs increase (a current positive trend in software engineering experimentation).The research reported in this short paper asks a simple and direct question: do we need to generate multiple sets of test cases? That is, how do different test sets influence the cost and effectiveness results? In a controlled experiment, we generated 10 different test cases to be adequate for the Statement Deletion (SSDL) mutation operator for 39 small programs and functions, and then evaluated how they differ in terms of cost and effectiveness. We found that averaging over multiple programs was effective in reducing the variance in the mutation scores introduced by specific tests.

show abstract

Empirical Evaluation of the Statement Deletion Mutation Operator

Cited by 61 publications

References 29 publications

Designing Deletion Mutation Operators

Designing Deletion Mutation Operators

Evaluation of Mutation Testing in a Nuclear Industry Case Study

Assessing the Influence of Multiple Test Case Selection on Mutation Experiments

Contact Info

Product

Resources

About