From Test Purposes to Asynchronous Test Cases

Abstract

In this paper, we investigate the problem of constructing a test case for a given test purpose and specification modelled by input/output transition systems (IOTS). The communication between the tester and the implementation under test is assumed to be asynchronous, performed via queues. Differently from synchronous tests, when issuing verdicts, asynchronous tests should take into account the distortion caused by the queues in the observed interactions. We propose an algorithm which constructs a sound test case, by transforming the test purpose prior to composing it with the specification without queues, mitigating the state explosion problem which usually occurs when queues are directly involved in the composition. Experimental results confirm the resulting state space reduction. The proposed approach can also be used to obtain sound asynchronous tests from synchronous tests. We identify a class of IOTS specifications for which synchronous and asynchronous tests coincide.

Synchrony and asynchrony in conformance testing

“…Hence, a subset of synchronous test-cases, namely those which only provide an input after observing quiescence, are safe for testing asynchronous systems. This is summarized in the following claim from [14,13] (and paraphrased in [7]):…”

“…In [7], the claim is taken for granted, and, unfortunately, in [14,13] only a proof sketch is provided for the above claim; the proof sketch is rather informal and leaves some room for interpretation, as illustrated by the following excerpt: "...An implementation guarantees that it will not send any output before receiving an input after quiescence is observed..."…”

“…In [14,7], it is argued that providing inputs to an IUT only after observing quiescence (i.e., in a stable state), eliminates the distortions in observable behavior, introduced by communicating to the IUT using queues. Hence, a subset of synchronous test-cases, namely those which only provide an input after observing quiescence, are safe for testing asynchronous systems.…”

“…In [7] a method is presented for generating test-cases from the synchronous specification that are sound for the asynchronous implementation. The main idea is to saturate a test-case with observation delays caused by asynchronous interactions.…”

“…The main idea is to saturate a test-case with observation delays caused by asynchronous interactions. In this paper, we adopt a restriction imposed on the implementation inspired by [7,Theorem 1] (dating back to [8]) and prove that in the setting of ioco testing this is sufficient for using synchronous test-case for the asynchronous implementation.…”