Improving logic-based testing

Kaminski, Gary; Ammann, Paul; Offutt, Jeff

doi:10.1016/j.jss.2012.08.024

Cited by 61 publications

(56 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kaminski et al [12] investigated a do-fewer approach. The relational operator replacement mutation operator (ROR) creates seven mutants per relational operator.…”

Section: Background and Related Workmentioning

confidence: 99%

Experimental Evaluation of SDL and One-Op Mutation for C

Delamaro

Deng

Durelli

et al. 2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

Abstract-Mutation analysis modifies a program by applying syntactic rules, called mutation operators, systematically to create many versions of the program (mutants) that differ in small ways. Testers then design tests to cause the mutants to behave differently from the original program. Mutation testing is widely considered to result in very effective tests, however, it is also quite costly. Cost comes from the many mutants that are created, the number of tests that are needed to kill the mutants, and the difficulty of deciding whether mutants behave equivalently to the original program. One-op mutation theorizes that cost can be reduced by using a single, very powerful, mutation operator that leads to tests that are almost as effective as if all operators are used. Previous research proposed the statement deletion operator (SDL) and found promising results. This paper investigates the use of SDL-mutation in a new context, the language C, and poses additional empirical questions, including whether other operators can be used. We carried out a controlled experiment in which cost and effectiveness of each individual C mutation operator were collected for 39 different subject programs. Experimental data are used to define a cost-effectiveness metric to choose the best single operator for one-op mutation.

show abstract

“…Kaminski et al [12] investigated a do-fewer approach. The relational operator replacement mutation operator (ROR) creates seven mutants per relational operator.…”

Section: Background and Related Workmentioning

confidence: 99%

Experimental Evaluation of SDL and One-Op Mutation for C

Delamaro

Deng

Durelli

et al. 2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Kaminski et al investigated a do-fewer approach theoretically [20]. The relational operator replacement mutation operator (ROR) generates seven mutants for every relational operator in a program.…”

Section: Background and Related Workmentioning

confidence: 99%

Empirical Evaluation of the Statement Deletion Mutation Operator

Deng

Offutt

2013

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

Abstract-Mutation analysis is widely considered to be an exceptionally effective criterion for designing tests. It is also widely considered to be expensive in terms of the number of test requirements and in the amount of execution needed to create a good test suite. This paper posits that simply deleting statements, implemented with the statement deletion (SDL) mutation operators in Mothra, is enough to get very good tests. A version of the SDL operator for Java was designed and implemented inside the muJava mutation system. The SDL operator was applied to 40 separate Java classes, tests were designed to kill the non-equivalent SDL mutants, and then run against all mutants.

show abstract

“…Offutt et al [35] found that focusing on replacing primitive operators sufficed to efficiently implement mutation testing for these languages. When implementing some of these operators, we can opt for (1) generating all possible mutations per mutation location, (2) producing a sufficient set of non-redundant mutations [36,37] or (3) introducing just one mutation in order to further reduce the cost. MuCPP implements option (1) for most of its operators, except for ARB, ROR, LOR and ASR, in which option (3) is implemented.…”

Section: Methodsmentioning

confidence: 99%

Assessment of class mutation operators for C++ with the MuCPP mutation system

Delgado‐Pérez

Medina‐Bulo

Palomo-Lozano

et al. 2017

Information and Software Technology

View full text Add to dashboard Cite

Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C++, despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C++ programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method: MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C++ programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C++. In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.

show abstract

Improving logic-based testing

Cited by 61 publications

References 34 publications

Experimental Evaluation of SDL and One-Op Mutation for C

Experimental Evaluation of SDL and One-Op Mutation for C

Empirical Evaluation of the Statement Deletion Mutation Operator

Assessment of class mutation operators for C++ with the MuCPP mutation system

Contact Info

Product

Resources

About