Pruning Testing Trees in the Test Template Framework by Detecting Mathematical Contradictions

Abstract

Abstract-Fastest is an automatic implementation of Phil Stocks and David Carrington's Test Template Framework (TTF) [1], a model-based testing (MBT) framework for the Z formal notation. In this paper we present a new feature of Fastest that helps TTF users to eliminate inconsistent test classes automatically. The method is very simple and practical, and makes use of the peculiarities of the TTF. Perhaps its most interesting features are extensibility and ease of use, since it does not assume previous knowledge on theorem proving. Also we compare the solution with a first attempt using the Z/EVES proof assistant and with the HOL-Z environment. At the end, we show the results of an empirical assessment based on applying Fastest to four real-world, industrial-strength case studies and to six toy examples.

Tool support for the Test Template Framework

“…This method was extensively described by Cristiá et al . . In Section 5.1, only its most salient features are shown, and in Section 5.2, how users should use it to make it more effective as more projects are run is shown.…”

“…As can be seen, all the contradictions are rather simple contradictory facts of set theory or integer arithmetic. When other experiments using other mathematical theories included in the Z mathematical toolkit (ZMT) are considered, contradictions relative to those theories might be found . Nevertheless, they are again rather simple contradictory facts of those theories.…”

“…On the other hand, as Cristiá et al . have described , a theorem prover can be used to guarantee the second point. In effect, they have certified with Z/EVES 50 of the 52 elimination theorems contained in the library at the moment the experiments shown in Section 9 were performed.…”

“…The results presented in this section were published in a previous paper ; they are included here so readers can find an integral presentation of the experiments carried out with Fastest. The philosophy behind the pruning method is that users should maintain the library of elimination theorems so the method becomes more automatic as more projects are executed.…”

“…This, in turn, enables the semi‐automation of other tasks, such as translating test cases into natural language (NL). In this paper, an integral presentation of previous work is made , and new results are introduced as well.…”