Experimental Evaluation of Mutation Testing Approaches to Python Programs

Derezińska, Anna; Hałas, Konrad

doi:10.1109/icstw.2014.24

Cited by 21 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Derezinska et al [42] proposed four algorithms to produce HOMs for Python programs. These algorithms are: Between-Operators, Each-Choice, FirstToLast, and Random.…”

Section: Related Workmentioning

confidence: 99%

An Approach for the Generation of Higher Order Mutants Using Genetic Algorithms

Abuljadayel¹,

Wedyan²

2018

IJISA

View full text Add to dashboard Cite

Abstract-Mutation testing is a structural testing technique in which the effectiveness of a test suite is measured by the suite ability to detect seeded faults. One fault is seeded into a copy of the program, called mutant, leading to a large number of mutants with a high cost of compiling and running the test suite against the mutants. Moreover, many of the mutants produce the same output as the original program (called equivalent mutants), such mutants need to be minimized to produce accurate results. Higher order mutation testing aims at solving these problems by allowing more than one fault to be seeded in the mutant. Recent work in higher order mutation show promising result in reducing the cost of mutation testing and increasing the approach effectiveness. In this paper, we present an approach for generating higher order mutants using a genetic algorithm. The aim of the proposed approach is to produce subtle and harder to kill mutants, and reduce the percentage of produced equivalent mutants. A Java tool has been developed, called HOMJava (Higher Order Mutation for Java), which implements the proposed approach. An experimental study was performed to evaluate the effectiveness of the proposed approach. The results show that the approach was able to produce subtle higher order mutants, the fitness of mutants improved by almost 99% compared with the first order mutants used in the experiment. The percentage of produced equivalent mutants was about 4%.

show abstract

“…Derezinska et al [42] proposed four algorithms to produce HOMs for Python programs. These algorithms are: Between-Operators, Each-Choice, FirstToLast, and Random.…”

Section: Related Workmentioning

confidence: 99%

An Approach for the Generation of Higher Order Mutants Using Genetic Algorithms

Abuljadayel¹,

Wedyan²

2018

IJISA

View full text Add to dashboard Cite

show abstract

“…PROTEUM/IM (Vincenzi et al, 2006) instruments the original code by using a descriptor language that assists in storing the nodes of the program graph reached by each test case, thus speeding up the mutant execution later. MutPy for Python (Derezinska & Halas, 2014) also implements a coverage analysis to reduce mutants generated and test cases executed. Bacterio (Mateo & Usaola, 2012) uses the mutant schemata technique (Untch et al, 1993) in order to reduce compilation times.…”

Section: Use Of Coverage Informationmentioning

confidence: 99%

“…Coverage information is usually collected in different mutation tools to reduce the computational cost of applying mutation testing (Vincenzi et al, 2006;Schuler & Zeller, 2009;Derezińska & Szustek, 2012;Mateo & Usaola, 2012;Derezinska & Halas, 2014), but we used this information to devise a new metric to measure operator effectiveness. Bacterio (Mateo & Usaola, 2012) retrieves coverage data to prevent unnecessary executions of mutants generated with both traditional and class operators for Java.…”

Section: Coveragementioning

confidence: 99%

“…Papadakis et al (2014) recently analysed the impact on the code coverage of mutations in order to classify mutants and mitigate the effects of equivalence. Thanks to the coverage facilities added to MutPy to reduce the mutants generated and test cases executed, Derezinska & Halas (2014) were able to discuss the impact of code coverage in mutation testing. They concluded that code coverage can significantly decrease the overall mutation time both in first and higher order mutants.…”

Section: Coveragementioning

confidence: 99%

See 1 more Smart Citation

Coverage-based quality metric of mutation operators for test suite improvement

2018

View full text Add to dashboard Cite

The choice of mutation operators is a fundamental aspect in mutation testing to guide the tester to an effective test suite. Designing a set of mutation operators is subject to a trade-off between effectiveness and computational cost: a larger mutation population might uncover more faults, but will take longer to analyse. With the aim of resolving this trade-off, several authors have defined an assortment of metrics to determine the most valuable operators. In this work, we extend an existing quality metric by incorporating an additional source of data, coverage information, and therefore investigate the extent to which mutants that often covered but rarely killed can improve the evaluation of mutation operators for the refinement of the test suite. As a case study, we analyse C++ class-level operators based on the new coveragebased quality metric to assess whether the original metric is enhanced. The results when selecting the best-valued operators show that this metric has great potential to help the tester in finding effective mutation operators. In comparison with the metric from which is derived, the use of coverage data allows to reduce the number of mutants but often losing fewer test cases and, in addition, retaining those that seem hard to design.

show abstract

“…By mining implementations directly from the IDE's version history rather than using final submissions, we capture a broader -and thus more interesting for testing -spectrum of buggy and incomplete solutions. Second, we ran mutpy [13], a mutation testing tool, on correct solutions to introduce a set of additional faulty implementations M, giving us an aggregate set of implementations A = S ∪ M. As a supplement to S, M serves two purposes. First, it introduces new test signatures for students to face as they develop their tests.…”

Section: Corpus Constructormentioning

confidence: 99%

An Automated System for Interactively Learning Software Testing

Smith

Tang

Warren

et al. 2017

Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

View full text Add to dashboard Cite

Testing is an important, time-consuming, and often difficult part of the software development process. It is therefore critical to introduce testing early in the computer science curriculum, and to provide students with frequent opportunities for practice and feedback. This paper presents an automated system to help introductory students learn how to test software. Students submit test cases to the system, which uses a large corpus of buggy programs to evaluate these test cases. In addition to gauging the quality of the test cases, the system immediately presents students with feedback in the form of buggy programs that nonetheless pass their tests. This enables students to understand why their test cases are deficient and gives them a starting point for improvement. The system has proven effective in an introductory class: students that trained using the system were later able to write better test cases-even without any feedback-than those who were not. Further, students reported additional benefits such as improved ability to read code written by others and to understand multiple approaches to the same problem.

show abstract

Experimental Evaluation of Mutation Testing Approaches to Python Programs

Cited by 21 publications

References 21 publications

An Approach for the Generation of Higher Order Mutants Using Genetic Algorithms

An Approach for the Generation of Higher Order Mutants Using Genetic Algorithms

Coverage-based quality metric of mutation operators for test suite improvement

An Automated System for Interactively Learning Software Testing

Contact Info

Product

Resources

About