Search-based mutation testing to improve performance tests

Sánchez, Ana María Sánchez; Delgado‐Pérez, Pedro; Medina‐Bulo, Inmaculada; Segura, Sergio

doi:10.1145/3205651.3205670

Cited by 3 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, the ratio of effective mutants may decrease as we add more traditional mutation operators, which is a problem taking into account the high cost of applying mutation testing. As a result of this issue, Sánchez et al [26] suggested the use of an evolutionary algorithm to guide the search towards useful traditional mutants for the refinement of performance tests.…”

Section: Traditional Mutation Operators To Address Non-functional Propertiesmentioning

confidence: 99%

“…In previous short papers [25,26], we have explored the potential benefits and limitations of the application of mutation testing at the performance level, what we called performance mutation testing (PMT). Analogously to classic mutation testing, the goal of PMT is to generate variants of the original program (performance-mutants), where each variant models a performance bug, for example, one that causes a noticeable degradation of the execution time of the program.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance mutation testing

Delgado‐Pérez

Sánchez

Segura

et al. 2020

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Performance bugs are known to be a major threat to the success of software products. Performance tests aim to detect performance bugs by executing the program through test cases and checking whether it exhibits a noticeable performance degradation. The principles of mutation testing, a well-established testing technique for the assessment of test suites through the injection of artificial faults, could be exploited to evaluate and improve the detection power of performance tests. However, the application of mutation testing to assess performance tests, henceforth called performance mutation testing (PMT), is a novel research topic with numerous open challenges. In previous papers, we identified some key challenges related to PMT. In this work, we go a step further and explore the feasibility of applying PMT at the source-code level in generalpurpose languages. To do so, we revisit concepts associated with classical mutation testing, and design seven novel mutation operators to model known bug-inducing patterns. As a proof of concept, we applied traditional mutation operators as well as performance mutation operators to open-source C++ programs. The results reveal the potential of the new performance-mutants to help assess and enhance performance tests when compared to traditional mutants. A review of live mutants in these programs suggests that they can induce the design of special test inputs. In addition to these promising results, our work brings a whole new set of challenges related to PMT, which will hopefully serve as a starting point for new contributions in the area.

show abstract

Section: Traditional Mutation Operators To Address Non-functional Propertiesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Performance mutation testing

Delgado‐Pérez

Sánchez

Segura

et al. 2020

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

show abstract

“…Despite the relatively large number of subjects in our data set, there are applications that require even larger numbers of defects to obtain statistically significant evaluation results. For these scenarios we envision the creation of performance mutants, along the lines proposed in recent work [10] and proposals [11], and provide insights to support the creation of performance mutants that resemble performance bugs fixed in real world projects.…”

Section: Introductionmentioning

confidence: 99%

Inferring Performance Bug Patterns from Developer Commits

Chen

Winter

Suri

2019

2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)

View full text Add to dashboard Cite

Performance bugs, i.e., program source code that is unnecessarily inefficient, have received significant attention by the research community in recent years. A number of empirical studies have investigated how these bugs differ from "ordinary" bugs that cause functional deviations and several approaches to aid their detection, localization, and removal have been proposed. Many of these approaches focus on certain subclasses of performance bugs, e.g., those resulting from redundant computations or unnecessary synchronization, and the evaluation of their effectiveness is usually limited to a small number of known instances of these bugs. To provide researchers working on performance bug detection and localization techniques with a larger corpus of performance bugs to evaluate against, we conduct a study of more than 700 performance bug fixing commits across 13 popular open source projects written in C and C++ and investigate the relative frequency of bug types as well as their complexity. Our results show that many of these fixes follow a small set of bug patterns, that they are contributed by experienced developers, and that the number of lines needed to fix performance bugs is highly project dependent.

show abstract