Software unit test coverage and adequacy

Zhu, Hong; Hall, Patrick A. V.; May, John H R

doi:10.1145/267580.267590

Cited by 955 publications

(726 citation statements)

References 177 publications

Supporting

Mentioning

679

Contrasting

Unclassified

Order By: Relevance

“…As mentioned in the introduction, it is assumed that M is correct, and its correctness has been assured, for example, by using model checking techniques [19] with respect to some properties derived from the requirements. A test generation algorithm Φ using a test selection criterion [20], for example, covering every node if M is graph-based, is applied to M such that Φ(M ) generates a test set T = {t 1 , t 2 , . .…”

Section: Modeling and Model-based Testingmentioning

confidence: 99%

“…A mutant is killed ("distinguished") by a test case t in T if the observed behavior of P differs from that of the mutant when executed against t. A mutant is equivalent to P if it always behaves the same as P for every case (i.e., every possible input). The test set T is said to be mutation adequate [20] with respect to P and X if every non-equivalent mutant of P generated by applying the mutation operators in X can be killed by at least one test case in T . Mutation testing has also been extended to validate a specification S. To do this, a set of mutation operators is applied to S to generate mutated specifications (S * ).…”

Section: Model-based Mutation Testing (Mbmt)mentioning

confidence: 99%

See 1 more Smart Citation

Model-based mutation testing—Approach and case studies

Belli

Budnik

Hollmann³

et al. 2016

Science of Computer Programming

View full text Add to dashboard Cite

This paper rigorously introduces the concept of model-based mutation testing (MBMT) and positions it in the landscape of mutation testing. Two elementary mutation operators, insertion and omission, are exemplarily applied to a hierarchy of graph-based models of increasing expressive power including directed graphs, event sequence graphs, finite-state machines and statecharts. Test cases generated based on the mutated models (mutants) are used to determine not only whether each mutant can be killed but also whether there are any faults in the corresponding system under consideration (SUC) developed based on the original model. Novelties of our approach are: (1) evaluation of the fault detection capability (in terms of revealing faults in the SUC) of test sets generated based on the mutated models, and (2) superseding of the great variety of existing mutation operators by iterations and combinations of the two proposed elementary operators. Three case studies were conducted on industrial and commercial real-life systems to demonstrate the feasibility of using the proposed MBMT approach in detecting faults in SUC, and to analyze its characteristic features. Our experimental data suggest that test sets generated based on the mutated models created by insertion operators are more effective in revealing faults in SUC than those generated by omission operators. Worth noting is that test sets following the MBMT approach were able to detect faults in the systems that were tested by manufacturers and independent testing organizations before they were released.

show abstract

Section: Modeling and Model-based Testingmentioning

confidence: 99%

Section: Model-based Mutation Testing (Mbmt)mentioning

confidence: 99%

Model-based mutation testing—Approach and case studies

Belli

Budnik

Hollmann³

et al. 2016

Science of Computer Programming

View full text Add to dashboard Cite

show abstract

“…This ratio is usually used as a decisive factor in determining the point in time at which to stop testing, that is, to release SUT, or to improve it and enhance the test set to continue testing [15]. To be more precise, the coverage C is defined as C = |O |/|O|, where O is a finite set of measuring objects, O is a subset of O that has been tested, and |O| represents the number of elements of O.…”

Section: N-switch Faulty Transition Coverage Criterionmentioning

confidence: 99%

Model-Based Mutation Testing Using Pushdown Automata

Belli

Beyazit

Takagi

et al. 2012

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYA model-based mutation testing (MBMT) approach enables to perform negative testing where test cases are generated using mutant models containing intentional faults. This paper introduces an alternative MBMT framework using pushdown automata (PDA) that relate to context-free (type-2) languages. There are two key ideas in this study. One is to gain stronger representational power to capture the features whose behavior depends on previous states of software under test (SUT). The other is to make use of a relatively small test set and concentrate on suspicious parts of the SUT by using MBMT approach. Thus, the proposed framework includes (1) a novel usage of PDA for modeling SUT, (2) novel mutation operators for generating PDA mutants, (3) a novel coverage criterion, and an algorithm to generate negative test cases from mutant PDA. A case study validates the approach, and discusses its characteristics and limitations.

show abstract

“…Testing being a complex and costly activity has motivated much research on efficient techniques to automate all aspects of the software testing process [3]. Of particular interest is the automation of test data generation [4] for functional unit testing, where the idea is to automatically generate a representative set of inputs for a unitary program fragment under test (typically a function or method). Running the considered code unit with the generated test data then offers a rather representative view of its actual behaviour, enabling one to detect errors efficiently.…”

Section: Introductionmentioning

confidence: 99%

“…In such an approach, the idea is to generate test data that in some way cover a sufficiently large part of the control-flow graph of the code unit under test [4]. In a nutshell, symbolic execution executes the unit under test over symbolic input values instead of concrete ones [5].…”

Section: Introductionmentioning

confidence: 99%

Relational symbolic execution of SQL code for unit testing of database programs

Marcozzi

Vanhoof

Hainaut

2015

Science of Computer Programming

View full text Add to dashboard Cite

Symbolic execution is a technique enabling the automatic generation of test inputs that exercise a set of execution paths within a code unit to be tested. If the paths cover a sufficient part of the code under test, the test data offer a representative view of the actual behaviour of this code. This notably enables detecting errors and correcting faults. Relational databases are ubiquitous in software, but symbolic execution of code units that manipulate them remains a non-trivial problem, particularly because of the complex structure of such databases and the complex behaviour of SQL statements. Finding errors in such code units is yet critical, as it can avoid corrupting important data. In this work, we define a symbolic execution translating database manipulation code directly into constraints and integrate it with a more traditional symbolic execution of normal program code. The database tables are represented by relational symbols and the SQL statements by relational constraints over these symbols. An algorithm based on these principles is presented for the symbolic execution of simple Java methods that implement transactional use cases by reading and writing in a relational database, the latter subject to data integrity constraints. The algorithm is integrated in a test generation tool and experimented over sample code. The target language for the constraints produced by the tool is the SMT-Lib standard and the used solver is Microsoft Z3. The results show that the proposed approach enables generating meaningful test data, including valid database content, in reasonable time. In particular, the Z3 solver is shown to be more scalable than the Alloy solver, used in our previous work, for solving relational constraints.

show abstract

Software unit test coverage and adequacy

Cited by 955 publications

References 177 publications

Model-based mutation testing—Approach and case studies

Model-based mutation testing—Approach and case studies

Model-Based Mutation Testing Using Pushdown Automata

Relational symbolic execution of SQL code for unit testing of database programs

Contact Info

Product

Resources

About