Experimental comparison of automated mutation testing tools for java

Rani, Shweta; Suri, Bharti; Khatri, Sunil Kumar

doi:10.1109/icrito.2015.7359265

Cited by 16 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar study was performed by Rani et al [28]. This study compared several Java mutation testing tools based on a set of manually generated test cases.…”

Section: Related Workmentioning

confidence: 90%

Analysing and Comparing the Effectiveness of Mutation Testing Tools: A Manual Study

Kintis

Papadakis

Papadopoulos

et al. 2016

2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM)

View full text Add to dashboard Cite

Mutation testing is considered as one of the most powerful testing methods. It operates by asking testers to design tests that reveal a set of mutants, which are purposemade injected defects. Evidently, the strength of the method strongly depends on the used mutants. However, this dependence raises concerns regarding the mutation testing practice that is implemented by existing tools. Thus, it is probable that implementation inadequacies can lead to incompetent results. In this paper, we cross-evaluate three popular mutation testing tools for Java, namely MUJAVA, MAJOR and PIT, with respect to their effectiveness. We perform an empirical study of 3,324 manually analysed mutants from real-world projects and we find that there are large differences between the tools' effectiveness, ranging from 76% to 88%, with MUJAVA achieving the best results. We also demonstrate that no tool is able to subsume the others and provide practical recommendations on how to strengthen each one of the studied tools. Finally, our analysis shows that 11%, 12% and 7% of the mutants generated by MUJAVA, MAJOR and PIT are equivalent, respectively.

show abstract

“…A similar study was performed by Rani et al [28]. This study compared several Java mutation testing tools based on a set of manually generated test cases.…”

Section: Related Workmentioning

confidence: 90%

Analysing and Comparing the Effectiveness of Mutation Testing Tools: A Manual Study

Kintis

Papadakis

Papadopoulos

et al. 2016

2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM)

View full text Add to dashboard Cite

show abstract

“…Parallel Isolated Test (PIT)-alternatively known as Pitest, is one of the best performing Mutation Testing tools for java that is well known for its fast process of mutants' generation [24]. The tool [25]- [27] is designed to conduct mutation analysis on java programs to support the software testers with the process of mutation testing on real-world codebases.…”

Section: Pit Toolmentioning

confidence: 99%

Investigation on Java Mutation Testing Tools

Park¹,

Hassan

Halim

et al. 2022

JOIV : Int. J. Inform. Visualization

View full text Add to dashboard Cite

Software Testing is one of the most significant phases within the software development life cycle since software bugs can be costly and traumatic. However, the traditional software testing process is not enough on its own as some undiscovered faults might still exist due to the test cases’ inability to detect all underlying faults. Amidst the various proposed techniques of test suites’ efficiency detection comes mutation testing, one of the most effective approaches as declared by many researchers. Nevertheless, there is not enough research on how well the mutation testing tools adhere to the theory of mutation or how well their mutation operators are performing the tasks they were developed for. This research paper presents an investigative study on two different mutation testing tools for Java programming language, namely PIT and µJava. The study aims to point out the weaknesses and strengths of each tool involved through performing mutation testing on four different open-source Java programs to identify the best mutation tool among them. The study aims to further identify and compare the mutation operators of each tool by calculating the mutation score. That is, the operators’ performance is evaluated with the mutation score, with the presumption that the more prominent the number of killed mutants is, the higher the mutation score, thus the more effective the mutation operator and the affiliated tool.

show abstract

“…Mutation testing is a fault-based testing technique and consider as white box testing for unit testing which provides a testing criterion called the mutation adequacy score. This mutation adequacy score can be used to measure the quality, reliability and performance effectiveness of a test set in terms of its ability to detect faults [1,4,5]. In the year of 1971, Richard Linkon et al first wrote a paper on fault tolerant of computer program [6].…”

Section: Mutationmentioning

confidence: 99%