Integrating Model-Based and Constraint-Based Testing Using SpecExplorer

Abstract

We report on our experience with model-based testing using SpecExplorer within the Flat X-Ray Detection (FXD) Department of Philips Healthcare. Our initial experiments showed a practical obstacle in combining traditional functional testing techniques with model-based testing using SpecExplorer. We overcome this obstacle by specifying the constraints on our data domain in a spreadsheet and interfacing SpecExplorer with a constraint solver in order to generate concrete test data for the behavioral specifications. We report on some empirical results obtained from our experiments.

Systematic mapping study on MBT: tools and models

“…Few studies were classified as Experimental Study, Proof of Concepts and Theoretical, representing, respectively, 9.2% (8), 3.45% (3) and 4.6% (4). From the perspective of MBT activities, the most researched MBT activity of the primary studies [27] Gherkin language [68] predicate/transition nets (Petri nets) [48] GUI model x [51, 69] RTCM [103] IF language [60] Simulink/Stateflow [41,77,84,89] Lyra modelling [25] TA mapping model [96] [30,40,55,90,92,102] model-implementation (MI) description [46] TTCN-3 [28,56] MI mapping) [48] Z notation [34,49] MSC [56] MCMs [35,37] Organization for the Advancement of Structured Information Standards (OASIS)/Open Service Oriented Architecture (OSOA) [21] Palladio component model [65] PBM [27] QML [20,23,104] SBM [74] SIM [74] S# test models [66] specification and abstraction language for testing [100] Spec# [9,23,42] TCM [74] TTSs [54] TestingMM [37] threat MI d...…”

On the Complexity of Input Output Conformance Testing

“…The main loop of Algorithm 1 iterates over the set of variables of the form X qp , for q ∈ Q and p ∈ S. This means there are at most |Q| × |S| iterations. The complexity of a single iteration is given by the sum of the complexity of the two inner loops (lines 9-17 and lines [19][20][21][22][23][24][25][26][27].…”

“…Testing is a major part of the software development process and, together with debugging, accounts for more than half of the development cost and effort [14]. Model-based testing is a structured and rigorous discipline of testing, which is likely to improve the current practice of testing [24,23,17]. Input-output conformance (ioco) testing is a well-known formal approach to model-based testing, which is used extensively in various practical applications, see [13,5] and the references therein, and which has been the subject of much theoretical research, see [21] and the references therein.…”

Using Formal Specifications to Support Model Based Testing ASDSpec: A Tool Combining the Best of Two Techniques

“…In contrast, MBT creates tests based on manually created test models, that verify actual implementations by executing sequences of commands. ASDSpec constructs basic test models based on ASD models, which can be used directly to create compliance tests for components or extended to describe requirements not covered by ASD verification, such as those that are data-testing related [16].…”

“…Furthermore, in addition to the verification ASD provides, Spec Explorer can use data testing to validate that the system implements desired behavior by linking inputs to observable events. Finally, Spec Explorer allows us to use model composition to test complete system, while also supporting data abstraction and combination [16] to reduce the state space to acceptable levels. In contrast, the facilities in ASD for handling data are very limited.…”