Detecting problematic message sequences and frequencies in distributed systems

Abstract

Testing the components of a distributed system is challenging as it requires consideration of not just the state of a component, but also the sequence of messages it may receive from the rest of the system or the environment. Such messages may vary in type and content, and more particularly, in the frequency at which they are generated. All of these factors, in the right combination, may lead to faulty behavior. In this paper we present an approach to address these challenges by systematically analyzing a component in a distributed system to identify specific message sequences and frequencies at which a failure can occur. At the core of the analysis is the generation of a test driver that defines the space of message sequences to be generated, the exploration of that space through the use of dynamic symbolic execution, and the timing and analysis of the generated tests to identify problematic frequencies. We implemented our approach in the context of the popular Robotic Operating System and investigated its application to three systems of increasing complexity.

Integration testing for robotic systems

“…The disadvantages are the need for a written test code and limited types of test based on input and output data or return time. Lucas et al (2012) described a component testing approach that analyzes sequences and frequencies of messages of components of a robotic distributed system running in ROS to identify failures. Weisshardt (2014) designed a test platform for robotic software components that provides different types of test, as simulation-in-the-loop, hardware-in-the-loop, and scenarios testing.…”

Distinguishing Sequences for Distributed Testing: Adaptive Distinguishing Sequences

“…If there is no global clock and the local testers do not synchronise during testing then we are testing in the ISO standardised distributed test architecture [17]. Sometimes, we allow the testers to exchange coordination messages through a network in order to synchronise their actions (see, for example, [32][33][34][35]). However, this can make testing more expensive, since it requires us to establish a network to connect the local testers, and may not be feasible where there are timing constraints.…”

Integration testing criteria for mobile robotic systems

“…The communication among these processes is established using the following topics: odometry, proximity, cmd motors, velocity, location, time To Impact and mapper activation. Listings 1 and 2 show two excerpts of the codes related to the Collision Avoidance [16] and Controller processes. The Controller process receives information from time to impact Callback topic (line 11) published by the Collision Avoidance process and processes it.…”