Test Criteria Definition for B Models

Abstract

Abstract. Test criteria are defined in order to guide the selection of subsets of the input domain to be covered during testing. A unification of two categories of test criteria, program based and specification based, i s presented. Such a unification is possible for B models because the specification, refinement concepts and implementation are captured in one notation. The notion of control flow graph is extended to handle the abstract constructs of the generalized substitution language, and a link between the coverage of the graph and the coverage of the before-after predicate i s established. A set of criteria for the coverage of the control flow graph i s proposed. These criteria are partially ordered according to their stringency, so that the coverage strategy may be tuned according to the complexity of the operation under test. IntroductionTesting is a partial verification technique that consists in exercising a target piece of software by supplying it with a sample of input values. Since exhaustive testing is generally not tractable, the tester is faced with the problem of selecting a proper subset of the input domain. The selection is guided by test criteria that specify a set of elements to be covered during testing. This paper focuses on the definition of test criteria for B models. It extends previous work establishing a validation framework for the B development process [13]. The aim is to track down specification faults originating from a misunderstanding of the functional requirements, or from the failure to adequately express an understood requirement. The B formal development process can be seen as a series of stages where more and more concrete models of the application are built, the final code being just a compiled version of the most concrete one. In order to validate these models, we have defined a uniform testing framework, irrespective of the development stage and of whether the test inputs are supplied to the final code or to the formal models [13]. Within the uniform framework, we wish to be equipped with coverage criteria that can be applied not only to the most abstract model (i.e. before refinement), but also to any intermediate model obtained during development. This must be so because, in typical B projects, the smallest meaningful model with respect to the functional requirements is likely to involve a few steps of refinement (see e.g. the modeling approach adopted

Using formal specifications to support testing

“…A unification of two categories of test criteria, using programbased and specification-based approaches, has been presented by Behnia and Waeselynck [Behnia and Waeselynck 1999] for B models. Vilkomir and Bowen have formalized control-flow testing criteria using the Z notation [Vilkomir and Bowen 2001;.…”

Mastering Test Generation from Smart Card Software Formal Models

“…The need to offer better methods and tools for functional testing has given rise to a large amount of research on generating tests from a formal model of the specifications, see for example [3][4][5][6][7]. Formal methods, and particularly model-oriented notations such as UML/OCL [8] and B [9] allow an abstract formalization of the expected behavior of the system under test.…”

Generation of test sequences from formal specifications: GSM 11‐11 standard case study

“…These notations are well suited for test generation because the expressiveness of set-oriented logic constructs and the definition of an explicit model help both test case generation and oracle synthesis. Thus, these formal notations are the basis of various proposals to more or less automatically generate tests from the formal model, see for example [7][8][9][10][11].In [12][13][14], we presented a new method for automated test generation from B abstract machines using constraint logic programming [15], called B-TESTING-TOOLS. From the B formal model of the functional requirements of the system to be tested, an equivalent system of constraints is derived and, then, each operation of the specification is partitioned through a translation into disjunctive normal form (DNF).…”

“…These notations are well suited for test generation because the expressiveness of set-oriented logic constructs and the definition of an explicit model help both test case generation and oracle synthesis. Thus, these formal notations are the basis of various proposals to more or less automatically generate tests from the formal model, see for example [7][8][9][10][11].…”