Formal passive testing of timed systems: theory and tools

Abstract

SUMMARY This paper presents a methodology to perform passive testing of timed systems. In passive testing, the tester does not interact with the implementation under test. On the contrary, execution traces are observed without interfering with the behaviour of the system. Invariants are used to represent the most relevant expected properties of the implementation under test. Intuitively, an invariant expresses the fact that each time the implementation under test performs a given sequence of actions, it must exhibit a behaviour in a lapse of time reflected in the invariant. There are two types of invariants: consequent and observational. The paper gives two algorithms to decide the correctness of proposed invariants with respect to a given specification and algorithms to check the correctness of a log, recorded from the implementation under test, with respect to an invariant. The soundness of this methodology is shown by relating it to an implementation relation. In addition to the theoretical framework, a tool called PASTE has been developed. This tool helps in the automation of the passive testing approach because it implements all the algorithms presented in this paper. PASTE takes advantage of mutation testing techniques in order to evaluate the goodness of an invariant according to its capability to detect errors in logs generated from mutants. An empirical study where PASTE was used to analyse a non‐trivial system is also reported. Copyright © 2012 John Wiley & Sons, Ltd.

Mutation Testing Advances: An Analysis and Survey

“…based on the number of killed mutants. Subsequent works detail (its mutation module, its mutant operators and the algorithms it incorporates), extend and evaluate the framework further [323,324] .…”

A survey on formal active and passive testing with applications to the cloud

“…In [5] it is presented a formal passive testing methodology for timed systems. The paper presents two algorithms to check the correctness of proposed invariants with respect to a given specification and algorithms to check the correctness of a log, recorded from the implementation under test, with respect to an invariant.…”

Runtime Verification Based on Executable Models: On-the-Fly Matching of Timed Traces

“…Let us denote the trace vertices (i.e., circles themselves) by y b , y a and y c (for the implementation) and x a , x b , x c and x d (for the specification). The implementation vertices are not causally related to each other, while the specification vertices are partially ordered (the precedence relation is drawn by arrows: x a ≺ x c , x b ≺ x c , x a ≺ x d and x b ≺ x d ) and are tagged with time intervals (δ (x a ) = [0, 2], δ (x b ) = [1,3], δ (x c ) = [0, 4] and δ (x d ) = [1,5]). Matchings are depicted by intermittent lines connecting the implementation vertices with the specification ones ((x a , y a ), (x b , y b ) and (x c , y c )).…”