Selecting mutation operators with a multiobjective approach

Banzi, Adam S.; Nobre, Tiago Henrique Souza; Pinheiro, Gabriel B.; Árias, João Carlos Garcia; Pozo, Aurora; Vergílio, Silvia Regina

doi:10.1016/j.eswa.2012.04.041

Cited by 17 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Adamopoulos et al [24] used a genetic algorithm for the co-evolution of mutant and test suite population, where difficult to kill mutants are favoured and equivalent mutants are penalised. Banzi et al [25] also applied a genetic algorithm but at the mutation operator level for selective mutation. They used a multi-objective approach to select mutation operators that maximise the adequacy of the test suite and minimise the number of mutants generated.…”

Section: Related Workmentioning

confidence: 99%

Using Evolutionary Mutation Testing to improve the quality of test suites

Delgado‐Pérez

Medina‐Bulo

Núñez

2017

2017 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

Mutation testing is a method used to assess and improve the fault detection capability of a test suite by creating faulty versions, called mutants, of the system under test. Evolutionary Mutation Testing (EMT), like selective mutation or mutant sampling, was proposed to reduce the computational cost, which is a major concern when applying mutation testing. This technique implements an evolutionary algorithm to produce a reduced subset of mutants but with a high proportion of mutants that can help the tester derive new test cases (strong mutants). In this paper, we go a step further in estimating the ability of this technique to induce the generation of test cases. Instead of measuring the percentage of strong mutants within the subset of generated mutants, we compute how much the test suite is actually improved thanks to those mutants. In our experiments, we have compared the extent to which EMT and the random selection of mutants help to find missing test cases in C++ objectoriented systems. We can conclude from our results that the percentage of mutants generated with EMT is lower than with the random strategy to obtain a test suite of the same size and that the technique scales better for complex programs.

show abstract

Section: Related Workmentioning

confidence: 99%

Using Evolutionary Mutation Testing to improve the quality of test suites

Delgado‐Pérez

Medina‐Bulo

Núñez

2017

2017 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

show abstract

“…While EMT follows a single-objective approach, several related works approach the selection of mutants as a multi-objective optimization problem. In this category, we can cite the work by Banzi et al [39], where a genetic algorithm is used for the selection of mutation operators instead of individual mutants. The authors of that study sought to maximize the mutation score and minimize the number of mutants generated.…”

Section: Related Workmentioning

confidence: 99%

Search-based mutant selection for efficient test suite improvement: Evaluation and results

Delgado‐Pérez

Medina‐Bulo

2018

Information and Software Technology

View full text Add to dashboard Cite

Context: Search-based techniques have been applied to almost all areas in software engineering, especially to software testing, seeking to solve hard optimization problems. However, the problem of selecting mutants to improve the test suite at a lower cost has not been explored to the same extent as other problems, such as mutant selection for test suite evaluation or test data generation. Objective: In this paper, we apply search-based mutant selection to enhance the quality of test suites efficiently. Namely, we use the technique known as Evolutionary Mutation Testing (EMT), which allows reducing the number of mutants while preserving the power to refine the test suite. Despite reported benefits of its application, the existing empirical results were derived from a limited number of case studies, a particular set of mutation operators and a vague measure, which currently makes it difficult to determine the real performance of this technique. Method: This paper addresses the shortcomings of previous studies, providing a new methodology to evaluate EMT on the basis of the actual improvement of the test suite achieved by using the evolutionary strategy. We make use of that methodology in new experiments with a carefully selected set of real-world C++ case studies. Results: EMT shows a good performance for most case studies and levels of demand of test suite improvement (around 45% less mutants than random selection in the best case). The results reveal that even a reduced subset of mutants selected with EMT can serve to increase confidence in the test suite, especially in programs with a large set of mutants. Conclusions: These results support the use of search-based techniques to solve the problem of mutant selection for a more efficient test suite refinement. Additionally, we identify some aspects that could foreseeably help enhance EMT.

show abstract

“…Barbosa et al (2001) and Namin et al (2008) tried to find sufficient sets of operators for C programs by defining a set of guidelines and a statistical analysis procedure respectively. Banzi et al (2012) also applied a genetic algorithm for the selection of mutation operators. They used a multi-objective approach: maximise the mutation score and minimise the number of mutants generated.…”

Section: Selective Mutationmentioning

confidence: 99%

Coverage-based quality metric of mutation operators for test suite improvement

2018

View full text Add to dashboard Cite

The choice of mutation operators is a fundamental aspect in mutation testing to guide the tester to an effective test suite. Designing a set of mutation operators is subject to a trade-off between effectiveness and computational cost: a larger mutation population might uncover more faults, but will take longer to analyse. With the aim of resolving this trade-off, several authors have defined an assortment of metrics to determine the most valuable operators. In this work, we extend an existing quality metric by incorporating an additional source of data, coverage information, and therefore investigate the extent to which mutants that often covered but rarely killed can improve the evaluation of mutation operators for the refinement of the test suite. As a case study, we analyse C++ class-level operators based on the new coveragebased quality metric to assess whether the original metric is enhanced. The results when selecting the best-valued operators show that this metric has great potential to help the tester in finding effective mutation operators. In comparison with the metric from which is derived, the use of coverage data allows to reduce the number of mutants but often losing fewer test cases and, in addition, retaining those that seem hard to design.

show abstract

Selecting mutation operators with a multiobjective approach

Cited by 17 publications

References 16 publications

Using Evolutionary Mutation Testing to improve the quality of test suites

Using Evolutionary Mutation Testing to improve the quality of test suites

Search-based mutant selection for efficient test suite improvement: Evaluation and results

Coverage-based quality metric of mutation operators for test suite improvement

Contact Info

Product

Resources

About