Property-based testing for the robot operating system

Abstract

The Robot Operating System (ROS) is an open source framework for the development of robotic software, in which a typical system consists of multiple processes communicating under a publishersubscriber architecture. A great deal of development time goes into orchestration and making sure that the communication interfaces comply with the expected contracts (e.g. receiving a message leads to the publication of another message). Orchestration mistakes are only detected during runtime, stressing the importance of component and integration testing in the verification process. Property-based Testing is fitting in this context, since it is based on the specification of contracts and treats tested components as black boxes, but there is no support for it in ROS. In this paper, we present a first approach towards automatic generation of test scripts for property-based testing of various configurations of a ROS system. CCS CONCEPTS• Software and its engineering → Software testing and debugging; Publish-subscribe / event-based architectures; • Computer systems organization → Robotics;

DiscoFuzzer: Discontinuity-based Vulnerability Detector for Robotic Systems

“…While previous work on property-based fuzzing is valid, Disco-Fuzzer outperforms them in detecting vulnerabilities that combine multiple properties in unison, which traditional property-based fuzzing detects less efficiently [39]. Additionally, DiscoFuzzer is more accessible for developers to validate discovered vulnerabilities.…”

ROS-Immunity: Integrated Approach for the Security of ROS-enabled Robotic Systems

“…The most critical component of a security tool is the ability to discover new vulnerabilities in a system. There have been several previous research endeavors to detect vulnerabilities for ROS systems, including static code analysis [24] [42], property-based testing [41], Honeypots [19] [29] , and fuzzing [18] [44]. All of these disjointed techniques yield valuable insights into the security of a robotic system, and as such, any integrated security system must make full use of them.…”

“…Indeed, an attacker can easily impersonate any subscriber node and querying the master for sensitive information. Santos et al [41] targeted ROS systems for a propertybased testing framework. The framework automatically generates test scripts for various configurations of a ROS system, while detecting for crashes or violation of defined properties.…”

DiscoFuzzer: Discontinuity-based Vulnerability Detector for Robotic Systems

“…A tester should design and implement every test in the proper language with test inputs and oracles for every behavior they want to check. Santo et al [39] extended ROS unit tests with property-based testing. Their tool randomly and automatically generates test inputs while checking two simple properties of the node: liveness (a node should not crash) and interface stability (the set of topics should not change).…”

“…However, Santos et alproposed a property-based testing approach for ROS. Property-based testing is very similar to fuzz testing, but the latter does not require familiarity with the target to specify properties and shapes [39].…”