A case study in model‐based testing of specifications and implementations

Abstract

SUMMARY Despite the existence of a number of animation tools for a variety of languages, methods for employing these tools for specification testing have not been adequately explored. Similarly, despite the close correspondence between specification testing and implementation testing, the two processes are often treated independently, and relatively little investigation has been performed to explore their relationship. This paper presents the results of applying a framework and method for the systematic testing of specifications and their implementations. This framework exploits the close correspondence between specification testing and implementation testing. The framework is evaluated on a sizable case study of the Global System for Mobile Communications 11.11 Standard, which has been developed towards use in a commercial application. The evaluation demonstrates that the framework is of similar cost‐effectiveness to the BZ‐Testing‐Tools framework and more cost‐effective than manual testing. A mutation analysis detected more than 95% of non‐equivalent specification and implementation mutants. Copyright © 2010 John Wiley & Sons, Ltd.

A taxonomy for requirements engineering and software test alignment

“…requirements, increases also the complexity of the software solution [Glass 2002], making therefore the testing more involved. Benefits of a strengthened link between RE and ST are, for example, improved product quality [Uusitalo et al 2008], cost-effective testing [Miller and Strooper 2010;Flammini et al 2009], high quality test-cases [de Santiago Júnior and Vijaykumar 2012], and early discovery of incomplete requirements [Siegl et al 2010].…”

“…To complement the description, we illustrate the application of the taxonomy by self-contained examples, based on Miller and Strooper [2010] case study on their framework and method of model-based testing of specifications and implementations. Note that section and figure numbers in the examples refer to Miller and Strooper [2010]. Finally, we apply the taxonomy on 13 alignment methods in Section 3.5.…”

Combining Centralised and Distributed Testing

“…This observation has led to interest in automating testing including work on model-based testing (MBT) in which testing is based on a model of the required behaviour of the system under test (SUT) or some aspect of this (see, for example, [En-Nouaary 2013;Farchi et al 2002;Grieskamp 2006;Hwang et al 2012;Miller and Strooper 2012;Tahat et al 2012;Tretmans 1996]). Given a model, there is the potential to automate test case generation and execution but also to automatically check that an observed behaviour is one allowed: the model acts as an Oracle.…”

What Do We Know about Alignment of Requirements Engineering and Software Testing?

“…The studies of this group (10 studies) supports alignment by facilitating test generation through formalization of the requirements. A formal specification gives a more precise and unambiguous description of the informal requirements [37]. This allows for formal verification of the requirements that can guarantee that the implementation conform to specification [38].…”