Planner based error recovery testing

Mayrhauser, A. von; Scheetz, M.; Dahlman, Eric; Howe, Adele E.

doi:10.1109/issre.2000.885871

Cited by 8 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Except for the entirely automatic test data generation, Baudry et al [106,107,108] focused on the automation of the test case enhancement phase: they optimised the test cases regarding mutation score via genetic and bacteriological algorithms, starting from an initial test suite. Von Mayrhauser et al [109] and Smith and Williams [110] augmented test input data using the requirement of killing as many mutants as possible. Compared with the existing literature survey of Hanh et al [71], which shed light on mutation-based test data generation, we cover more studies and extend their work to 2015.…”

Section: Attribute Framework Validationmentioning

confidence: 99%

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s, and remarkable advances have been achieved in its concepts, theory, technology, and empirical evidence.While the most influential realisations have been summarised by existing literature reviews, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 191 papers published between 1981 and 2015. In particular, we analysed in which quality assurance processes mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the inherent core problems of mutation testing, ie, the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped.Based on our observations, we made 9 recommendations for future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner. KEYWORDSapplication, mutation testing, systematic literature review 1 INTRODUCTION Mutation testing is defined by Jia and Harman [1] as a fault-based testing technique, which provides a testing criterion called the mutation adequacy score. This score can be used to measure the effectiveness of a test set in terms of its ability to detect faults [1]. The principle of mutation testing is to introduce syntactic changes into the original program to generate faulty versions (called mutants) according to well-defined rules (mutation operators) [2]. Mutation testing originated in the 1970s with works from Lipton [3], DeMillo et al. [4], and Hamlet [5] and has been a very active research field over the last few decades. The activeness of the field is in part evidenced by the extensive survey of more than 390 papers on mutation testing that Jia and Harman published in 2011 [1]. Jia and Harman's survey highlights the research achievements that have been made over the years, including the development of tools for a variety of languages and empirical studies performed [1]. Additionally, they highlight some of the actual and inherent problems of mutation testing, among others: (1) the high computational cost caused by generating and executing the numerous mutants and (2) the tremendous time-consuming human investigation required by the test oracle problem and equivalent mutant detection.While existing surveys (eg [1,2,6]) provide us with a great overview of the most influential realisations in research, we lack insight into how mutation testing is actually applied. Specifically, we are interested in analysing ...

show abstract

Section: Attribute Framework Validationmentioning

confidence: 99%

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…Except for the entirely automatic test data generation, Baudry et al [92][93][94] focused on the automation of the test case enhancement phase: they optimised the test cases regarding mutation score via genetic and bacteriological algorithms, starting from an initial test suite. von Mayrhauser et al [95] and Simith and Williams [96] augmented test input data using the requirement of killing as many mutants as possible.…”

Section: Rq11 and Rq12: Which Role Does Mutation Testing Play In Eachmentioning

confidence: 99%

“…We did find that only 31studies use medium size subjects, which corresponds to 19% of cases. Finally, we observed only two cases where mutation testing is applied in more large-scale projects: (i) Qi et al [99] adopted mutation testing to prioritise test cases to speed up patch validation during program repairing and (ii) von Mayrhauser et al [95] proposed a new test data generation technique based on an Artificial Intelligence (AI) planner and evaluated their method on an Automated Cartridge System (ACS). These two instances show the full potential of mutation testing to be employed as a practical testing tool for large industrial systems.…”

Section: Rq25: What Are the Most Common Subjects Used In The Experimmentioning

confidence: 99%

A systematic literature review of how mutation testing supports test activities

Zhu

Panichella

Zaidman

2016

Preprint

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s and remarkable advances have been achieved in its concepts, theory, technology and empirical evidence. While the latest realisations have been summarised by existing literature review, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 159 papers published between 1981 and 2015. In particular, we analysed in which testing activities mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the core inherent problems of mutation testing, i.e. the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped. Based on our observations, we made nine recommendations for the future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner.

show abstract

“…A more active area of research since the mid-1990s has been the use of AI planning for testing. von Mayrhauser, Scheetz, Dahlman & Howe (2000) point out that a major disadvantage of white box testing is that we have to wait until the code is developed before commencing the process of producing the tests. An alternative to the use of white-box testing is to model the domain and produce tests from the model.…”

Section: Ai Planningmentioning

confidence: 99%

Artificial Intelligence Applications for Improved Software Engineering Development

2010

Advances in Computational Intelligence and Robotics

View full text Add to dashboard Cite

Planner based error recovery testing

Cited by 8 publications

References 24 publications

A systematic literature review of how mutation testing supports quality assurance processes

A systematic literature review of how mutation testing supports quality assurance processes

A systematic literature review of how mutation testing supports test activities

Artificial Intelligence Applications for Improved Software Engineering Development

Contact Info

Product

Resources

About