Combining Centralised and Distributed Testing

Abstract

Many systems interact with their environment at distributed interfaces (ports) and sometimes it is not possible to place synchronised local testers at the ports of the system under test (SUT). There are then two main approaches to testing: having independent local testers or a single centralised tester that interacts asynchronously with the SUT. The power of using independent testers has been captured using implementation relation dioco. In this paper we define implementation relation diococ for the centralised approach and prove that dioco and diococ are incomparable. This shows that the frameworks detect different types of faults and so we devise a hybrid framework and define an implementation relation diocos for this. We prove that the hybrid framework is more powerful than the distributed and centralised approaches. We then prove that the Oracle problem is NP-complete for diococ and diocos but can be solved in polynomial time if we place an upper bound on the number of ports. Finally, we consider the problem of deciding whether there is a test case that is guaranteed to force a finite state model into a particular state or to distinguish two states, proving that both problems are undecidable for the centralised and hybrid frameworks.

A More Precise Implementation Relation for Distributed Testing

“…. ∩ L m (σ m ) in polynomial time; a similar result has been shown for the corresponding problem when testing using dioco and IOTSs are not output-divergent [44].…”

“…FSMs differ significantly from IOTSs since, for example, input and output alternate and an FSM is quiescent after an output has been produced. A similar result has been shown for the corresponding problem when testing using dioco and IOTSs but is restricted to the case where the IOTSs are not output-divergent and also where the observation is a quiescent trace and not a tuple of local traces [44].…”

Towards Decentralized Conformance Checking in Model-Based Testing of Distributed Systems

“…In order to be able to check not only the interactions with the environment but also the interactions between the components of the system under test (SUT), we follow a hybrid test architecture, in which a local tester is deployed close to each system component, and a central tester coordinates the local testers (see Figure 1). This is more effective than a purely centralized approach or a purely distributed approach, meaning that more conformance errors can be detected [7].…”

“…One difficulty in testing distributed systems is that their distributed nature imposes theoretical limitations on the conformance faults that can be detected by the test components, depending on the test architecture used [7], [21]. Two basic test architectures have been proposed in the past to test distributed systems: a purely distributed test architecture with independent local testers communicating synchronously with the components of the SUT [22]; a purely centralized test architecture, in which a single centralized tester interacts asynchronously with the components of the SUT.…”

“…Two basic test architectures have been proposed in the past to test distributed systems: a purely distributed test architecture with independent local testers communicating synchronously with the components of the SUT [22]; a purely centralized test architecture, in which a single centralized tester interacts asynchronously with the components of the SUT. More recently, Hierons [7] proposed a hybrid framework that combines local testers and a centralized tester. He proved that this architecture is more powerful than the distributed and centralized approaches, i.e., it has a higher fault detection capability.…”