LittleDarwin: A Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems

The state-of-the-practice in software development is driven by constant change fueled by continuous integration servers. Such constant change demands for frequent and fully automated tests capable to detect faults immediately upon project build. As the fault detection capability of the test suite becomes so important, modern software development teams continuously monitor the quality of the test suite as well. However, it appears that the state-of-the-practice is reluctant to adopt strong coverage metrics (namely mutation coverage), instead relying on weaker kinds of coverage (namely branch coverage). In this paper, we investigate three reasons that prohibit the adoption of mutation coverage in a continuous integration setting: (1) the difficulty of its integration into the build system, (2) the perception that branch coverage is "good enough", and (3) the performance overhead during the build. Our investigation is based on a case study involving four open source systems and one industrial system. We demonstrate that mutation coverage reveals additional weaknesses in the test suite compared to branch coverage, and that it is able to do so with an acceptable performance overhead during project build.

show abstract

“…PITest is used in this study in an attempt to demonstrate the feasibility of mutation testing in an industrial environment (RQ1). [78].…”

Section: Pitestmentioning

confidence: 99%

Comparing mutation coverage against branch coverage in an industrial setting

Parsai

Demeyer

2020

Int J Softw Tools Technol Transfer

Self Cite

View full text Add to dashboard Cite

show abstract

“…Mutation Testing with LittleDarwin. LittleDarwin is a Java mutation testing tool that mainly targets fast deployment in complicated software environments [6]. In this hands-on session, the participants used LittleDarwin to perform mutation testing on an educational software project, and by following the step-by-step tutorial they familiarized themselves with the process of mutation testing.…”

Section: A-test 2018mentioning

confidence: 99%

Automating TEST Case Design, Selection and Evaluation Report on 10 Editions of A-TESTWorkshop

Vos

Prasetya

Eldh

et al. 2020

SIGSOFT Softw. Eng. Notes

Self Cite

View full text Add to dashboard Cite

Trends such as globalisation, standardisation and shorter life-cycles place great demands on the flexibility of the software industry. In order to compete and cooperate on an international scale, a constantly decreasing time to market and an increasing level of quality are essential. Testing is at the moment the most important and mostly used quality assurance technique applied in industry. However, the complexity of software and hence of their development amount is increasing. Modern systems get larger and more complex, as they connect large amounts of components that interact in many different ways and have constantly changing and different types of requirements (functionality, dependability, usability, performance etc.). Data processing that impacts all aspects of our life is increasingly distributed over clouds and devices. This leads to new concerns, such as availability, security, and privacy, which are aspects that also needs to be tested. Consequently, the development of cost-effective and high-quality systems opens new challenges that cannot be faced only with traditional testing approaches, and specifically manual testing is simply insufficient and unreliable to manage the speed needed, and ensure the coverage of ever-changing systems. New techniques for systematization and automation of testing throughout the software and system life-cycle are required.

show abstract

“…However, the tool is not publicly available. More recently, LittleDarwin [21], a higher order mutation tool for Java, was released, although the authors report longer mutation analysis run-times than with PIT. We were unable to find precise documentation regarding its HOM features on its website and to compare it to our tool.…”

Section: Background and Related Workmentioning

confidence: 99%

PIT-HOM: an Extension of Pitest for Higher Order Mutation Analysis

Laurent

Ventresque

2019

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

View full text Add to dashboard Cite

Mutation testing is a well-known, effective, faultbased testing criterion. First order mutation introduces defects in the form of a single small syntactic change. While the technique has been shown to be effective, it has some limits. Higher order mutation, where the faults introduced include multiple changes, has been proposed as a way to address some of these limits.Although the technique has shown promising results, there is no practical tool available for the application and study of higher order mutation on Java programs.In this paper we present PIT-HOM, an extension of Pitest (PIT) for higher order mutation. Pitest is a practical mutation analysis tool for Java, applicable on real-world codebases. PIT-HOM combines mutants in a same class to create higher order mutants of user-defined orders, it runs the mutants and reports the results in an easy to process format. We validate PIT-HOM using two small Java programs and report its performance as well as some characteristics of the mutants it creates.

show abstract

LittleDarwin: A Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems

Cited by 19 publications

References 35 publications

Comparing mutation coverage against branch coverage in an industrial setting

Comparing mutation coverage against branch coverage in an industrial setting

Automating TEST Case Design, Selection and Evaluation Report on 10 Editions of A-TESTWorkshop

PIT-HOM: an Extension of Pitest for Higher Order Mutation Analysis

Contact Info

Product

Resources

About