Selecting a software engineering tool: lessons learnt from mutation analysis

Delahaye, Mickaël; Bousquet, Lydie Du

doi:10.1002/spe.2312

Cited by 24 publications

(39 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these tools, Proteum/IM 2.0 [38] and MILU [39] are the most widely used in other research studies, such as to evaluate selective mutation [14] or higher order mutation [12]. While MILU offers fewer features than Proteum, it automates most of the mutation analysis process, in contrast to Proteum which requires considerable manual intervention [40].…”

Section: F Mutation Toolmentioning

confidence: 99%

“…The survey by Delahaye and du Bousquet [40] states that MILU implements the 77 mutation operators that Agrawal et al [17] identified for C. However, the number of mutation operators has been reduced in MILU 3.2 -following studies on SM [13], [14], only 12 of those 77 operators were included in this version. Two additional operators are also included: SSDL (delete statements) and SBRC (replaces break by continue).…”

Section: G Cost Reduction Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

Section: F Mutation Toolmentioning

confidence: 99%

Section: G Cost Reduction Techniquesmentioning

confidence: 99%

Evaluation of Mutation Testing in a Nuclear Industry Case Study

et al. 2018

View full text Add to dashboard Cite

“…Among the Java tools, the most popular ones are the MuJava [7] and the Major [5]. Unfortunately, these tools were built to support research projects and thus, their practical use is limited [3]. In short PIT offers the following three major advantages over the other tools: a) it is open source, b) it is well integrated with development tools, as it offers a Maven plugin and c) it is quite robust and actively maintained.…”

Section: Mutation Testingmentioning

confidence: 99%

“…PIT also operates on the latest version of Java. For further details regarding the tool please refer to the work of Delahaye et al who compared the existing Java mutation testing tools [3].…”

Section: Mutation Testingmentioning

confidence: 99%

“…PIT is considerably fast as it manipulates bytecode and runs only the tests that have a chances to kill the used mutants (i.e., the tests that execute the instruction where the mutant is located). PIT's major advantage is that it is robust, easy to use and well integrated with development tools [3]. The present paper aims at describing the tool along with its latest improvements on the supported mutants 1 .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PIT: a practical mutation testing tool for Java (demo)

Coles

Laurent

Henard

et al. 2016

Proceedings of the 25th International Symposium on Software Testing and Analysis

198

156

View full text Add to dashboard Cite

Mutation analysis introduces program defects with the intend of verifying whether candidate tests are able to trigger anomalous behaviour. In case the tests can distinguish the defective behaviour from that of the original program, they are considered of good quality -otherwise developers need to design new tests. While, this method has been shown to be effective, industry-scale code challenges its applicability due to the sheer number of mutants and test executions it requires. In this paper we present PIT, a practical mutation testing tool for Java, applicable on real-world codebases. PIT is fast since it operates on bytecode and optimises mutant executions. It is also robust and well integrated with development tools, as it can be invoked through a command line interface, Ant or Maven. PIT is also open source and hence, publicly available at http://pitest.org/

show abstract

PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systems

et al. 2017

View full text Add to dashboard Cite

The identification of performance issues and the diagnosis of their root causes are time-consuming and complex tasks, especially in clustered environments. To simplify these tasks, researchers have been developing tools with built-in expertise for practitioners. However, various limitations exist in these tools that prevent their efficient usage in the performance testing of clusters (e.g. the need of manually analysing huge volumes of distributed results). In a previous work, we introduced a policy-based adaptive framework (PHOEBE) that automates the usage of diagnosis tools in the performance testing of clustered systems, in order to improve a tester's productivity, by decreasing the effort and expertise needed to effectively use such tools. This paper extends that work by broadening the set of policies available in PHOEBE, as well as by performing a comprehensive assessment of PHOEBE in terms of its benefits, costs and generality (with respect to the used diagnosis tool). The performed evaluation involved a set of experiments in assessing the different trade-offs commonly experienced by a tester when using a performance diagnosis tool, as well as the time savings that PHOEBE can bring to the performance testing and analysis processes. Our results have shown that PHOEBE can drastically reduce the effort required by a tester to do performance testing and analysis in a cluster. PHOEBE also exhibited consistent behaviour (i.e. similar time-savings and resource utilisations), when applied to a set of commonly used diagnosis tools, demonstrating its generality. Finally, PHOEBE proved to be capable of simplifying the configuration of a diagnosis tool. This was achieved by addressing the identified trade-offs without the need for manual intervention from the tester. PHOEBE is implemented with the multi-agent architecture depicted in Figure 1. There, it can be seen how PHOEBE is composed of three types of agents: the control agent is responsible for interacting with the load testing tool to know when the test starts and ends. It is also responsible for evaluating the policies and propagating the decisions to the other nodes. Meanwhile, the application node agent is responsible for performing the required tasks in each application node (e.g. sampling collection or sending the collected samples to the diagnosis tool). Finally, the diagnosis tool Here, the objective was to evaluate the potential trade-off between the number of samples concurrently processed by a diagnosis tool and the number of resources it requires to process the samples. The following sections describe this experiment and its results. PHOEBE 1863 7.1. Experiment #4: proposed policies assessmentThe objective of this experiment was to evaluate the behaviour of PHOEBE, as well as the set of proposed policies, in order to assess how well they have fulfilled their purpose of addressing the identified trade-offs without the need for manual intervention from the tester. The following sections describe this experiment and its results.7.1.1. Experimental set-up....

show abstract

Selecting a software engineering tool: lessons learnt from mutation analysis

Cited by 24 publications

References 38 publications

Evaluation of Mutation Testing in a Nuclear Industry Case Study

Evaluation of Mutation Testing in a Nuclear Industry Case Study

PIT: a practical mutation testing tool for Java (demo)

PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systems

Contact Info

Product

Resources

About