Behaviour abstraction adequacy criteria for API call protocol testing

Czemerinski, Hernán; Braberman, Vı́ctor; Uchitel, Sebastián

doi:10.1002/stvr.1593

Cited by 3 publications

(10 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…al [54] on automatic mining of EPA models, while five subjects were taken from the article by Czemerinski et. al [28]. These subjects also appear in other papers (e.g., [24,33,56,74]).…”

Section: Mutation Study On Subjects With Third Party Epasmentioning

confidence: 65%

“…As previously mentioned, Czemerinski et al [28] proposed a conformance testing adequacy criteria based on covering EPA transitions. Given an EPA for a target class, the EPA transition coverage for a test suite T S is defined as follows:…”

Section: Epa Transition Coveragementioning

confidence: 99%

“…Czemerinski et al proposed a conformance testing adequacy criteria based on covering EPAs [28]. They showed a correlation between transition coverage in the EPA of the intended object protocol and the ability of test suites to detect protocol conformance failures.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Enabledness-based Testing of Object Protocols

Godoy

Galeotti

Garbervetsky

et al. 2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

A significant proportion of classes in modern software introduce or use object protocols, prescriptions on the temporal orderings of method calls on objects. This article studies search-based test generation techniques that aim to exploit a particular abstraction of object protocols (enabledness preserving abstractions (EPAs)) to find failures. We define coverage criteria over an extension of EPAs that includes abnormal method termination and define a search-based test case generation technique aimed at achieving high coverage. Results suggest that the proposed case generation technique with a fitness function that aims at combined structural and extended EPA coverage can provide better failure-detection capabilities not only for protocol failures but also for general failures when compared to random testing and search-based test generation for standard structural coverage.

show abstract

“…al [54] on automatic mining of EPA models, while five subjects were taken from the article by Czemerinski et. al [28]. These subjects also appear in other papers (e.g., [24,33,56,74]).…”

Section: Mutation Study On Subjects With Third Party Epasmentioning

confidence: 65%

Section: Epa Transition Coveragementioning

confidence: 99%

See 1 more Smart Citation

Enabledness-based Testing of Object Protocols

Godoy

Galeotti

Garbervetsky

et al. 2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…Among the papers in our set, we also found 19 studies that performed mutant analysis, which means that the researchers tried to get a better understanding about mutation faults, eg, which faults are more valuable in a particular context. A good example of this mutant analysis is the hard-mutant problem investigated by Czemerinski et al [105] where they analysed the failure rate for the hard-to-kill mutants (killed by less than 20% of test cases) using the domain partition approach.…”

Section: Attribute Framework Validationmentioning

confidence: 99%

A systematic literature review of how mutation testing supports quality assurance processes

Zhu

Panichella

Zaidman

2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s, and remarkable advances have been achieved in its concepts, theory, technology, and empirical evidence.While the most influential realisations have been summarised by existing literature reviews, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 191 papers published between 1981 and 2015. In particular, we analysed in which quality assurance processes mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the inherent core problems of mutation testing, ie, the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped.Based on our observations, we made 9 recommendations for future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner. KEYWORDSapplication, mutation testing, systematic literature review 1 INTRODUCTION Mutation testing is defined by Jia and Harman [1] as a fault-based testing technique, which provides a testing criterion called the mutation adequacy score. This score can be used to measure the effectiveness of a test set in terms of its ability to detect faults [1]. The principle of mutation testing is to introduce syntactic changes into the original program to generate faulty versions (called mutants) according to well-defined rules (mutation operators) [2]. Mutation testing originated in the 1970s with works from Lipton [3], DeMillo et al. [4], and Hamlet [5] and has been a very active research field over the last few decades. The activeness of the field is in part evidenced by the extensive survey of more than 390 papers on mutation testing that Jia and Harman published in 2011 [1]. Jia and Harman's survey highlights the research achievements that have been made over the years, including the development of tools for a variety of languages and empirical studies performed [1]. Additionally, they highlight some of the actual and inherent problems of mutation testing, among others: (1) the high computational cost caused by generating and executing the numerous mutants and (2) the tremendous time-consuming human investigation required by the test oracle problem and equivalent mutant detection.While existing surveys (eg [1,2,6]) provide us with a great overview of the most influential realisations in research, we lack insight into how mutation testing is actually applied. Specifically, we are interested in analysing ...

show abstract

“…which faults are more valuable in a particular context. A good example of this mutant analysis is the hard mutant problem investigated by Czemerinski et al [91] where they analysed the failure rate for the hard-to-kill mutants (killed by less than 20% of test cases) using the domain partition approach.…”

Section: Rq11 and Rq12: Which Role Does Mutation Testing Play In Eachmentioning

confidence: 99%

A systematic literature review of how mutation testing supports test activities

Zhu

Panichella

Zaidman

2016

Preprint

View full text Add to dashboard Cite

Mutation testing has been very actively investigated by researchers since the 1970s and remarkable advances have been achieved in its concepts, theory, technology and empirical evidence. While the latest realisations have been summarised by existing literature review, we lack insight into how mutation testing is actually applied. Our goal is to identify and classify the main applications of mutation testing and analyse the level of replicability of empirical studies related to mutation testing. To this aim, this paper provides a systematic literature review on the application perspective of mutation testing based on a collection of 159 papers published between 1981 and 2015. In particular, we analysed in which testing activities mutation testing is used, which mutation tools and which mutation operators are employed. Additionally, we also investigated how the core inherent problems of mutation testing, i.e. the equivalent mutant problem and the high computational cost, are addressed during the actual usage. The results show that most studies use mutation testing as an assessment tool targeting unit tests, and many of the supporting techniques for making mutation testing applicable in practice are still underdeveloped. Based on our observations, we made nine recommendations for the future work, including an important suggestion on how to report mutation testing in testing experiments in an appropriate manner.

show abstract

Behaviour abstraction adequacy criteria for API call protocol testing

Cited by 3 publications

References 85 publications

Enabledness-based Testing of Object Protocols

Enabledness-based Testing of Object Protocols

A systematic literature review of how mutation testing supports quality assurance processes

A systematic literature review of how mutation testing supports test activities

Contact Info

Product

Resources

About