Higher Order Mutation Testing

Jia, Yunyi; Harman, Mark

doi:10.1016/j.infsof.2009.04.016

Cited by 237 publications

(219 citation statements)

References 37 publications

Supporting

Mentioning

216

Contrasting

Unclassified

Order By: Relevance

“…Several techniques have been proposed to address this issue [4]. Some of them can be classified as "do fewer" techniques, in the sense that they seek to reduce the number of mutants, such as random mutant selection [11] (i.e., sampling a percentage of the full set of mutants) and higher order mutation [12] (i.e., several mutations are combined into a single mutant). Selective mutation, perhaps the cost reduction technique with the greatest acceptance [13], [14], [15], [16], works under the assumption that some mutation operators can be excluded without sacrificing a great deal of fault-revealing power.…”

Section: Improving the Efficiency Of Mutation Testingmentioning

confidence: 99%

“…While several of them are commercial 1 or are not publicly available, five of them are accessible. Among these tools, Proteum/IM 2.0 [38] and MILU [39] are the most widely used in other research studies, such as to evaluate selective mutation [14] or higher order mutation [12]. While MILU offers fewer features than Proteum, it automates most of the mutation analysis process, in contrast to Proteum which requires considerable manual intervention [40].…”

Section: F Mutation Toolmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

Section: Improving the Efficiency Of Mutation Testingmentioning

confidence: 99%

Section: F Mutation Toolmentioning

confidence: 99%

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

“…Several techniques have been proposed to generate fewer mutants, including mutant sampling [44], high-order mutation [45] and mutant clustering [46]. Another common technique for mutant reduction is selective mutation, which has been mainly applied to procedural languages like FORTRAN [27] or Ada [38].…”

Section: Related Workmentioning

confidence: 99%

Assessment of class mutation operators for C++ with the MuCPP mutation system

Delgado‐Pérez

Medina‐Bulo

Palomo-Lozano

et al. 2017

Information and Software Technology

View full text Add to dashboard Cite

Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C++, despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C++ programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method: MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C++ programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C++. In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.

show abstract

“…3) can help us detect faults in SUC. Also, one may argue that in addition to the first-order mutants, higher-order mutants should be used as well because they might be more appropriate to simulate real faults [34,35]. However, the disadvantage is that this will also increase the number of mutants and the cost of mutation testing significantly.…”

Section: Threats To Validitymentioning

confidence: 99%

Model-based mutation testing—Approach and case studies

Belli

Budnik

Hollmann³

et al. 2016

Science of Computer Programming

View full text Add to dashboard Cite

This paper rigorously introduces the concept of model-based mutation testing (MBMT) and positions it in the landscape of mutation testing. Two elementary mutation operators, insertion and omission, are exemplarily applied to a hierarchy of graph-based models of increasing expressive power including directed graphs, event sequence graphs, finite-state machines and statecharts. Test cases generated based on the mutated models (mutants) are used to determine not only whether each mutant can be killed but also whether there are any faults in the corresponding system under consideration (SUC) developed based on the original model. Novelties of our approach are: (1) evaluation of the fault detection capability (in terms of revealing faults in the SUC) of test sets generated based on the mutated models, and (2) superseding of the great variety of existing mutation operators by iterations and combinations of the two proposed elementary operators. Three case studies were conducted on industrial and commercial real-life systems to demonstrate the feasibility of using the proposed MBMT approach in detecting faults in SUC, and to analyze its characteristic features. Our experimental data suggest that test sets generated based on the mutated models created by insertion operators are more effective in revealing faults in SUC than those generated by omission operators. Worth noting is that test sets following the MBMT approach were able to detect faults in the systems that were tested by manufacturers and independent testing organizations before they were released.

show abstract

Higher Order Mutation Testing

Cited by 237 publications

References 37 publications

Evaluation of Mutation Testing in a Nuclear Industry Case Study

Evaluation of Mutation Testing in a Nuclear Industry Case Study

Assessment of class mutation operators for C++ with the MuCPP mutation system

Model-based mutation testing—Approach and case studies

Contact Info

Product

Resources

About