ros2_tracing: Multipurpose Low-Overhead Framework for Real-Time Tracing of ROS 2

Abstract

Testing and debugging have become major obstacles for robot software development, because of high system complexity and dynamic environments. Standard, middlewarebased data recording does not provide sufficient information on internal computation and performance bottlenecks. Other existing methods also target very specific problems and thus cannot be used for multipurpose analysis. Moreover, they are not suitable for real-time applications. In this paper, we present ros2 tracing, a collection of flexible tracing tools and multipurpose instrumentation for ROS 2. It allows collecting runtime execution information on real-time distributed systems, using the low-overhead LTTng tracer. Tools also integrate tracing into the invaluable ROS 2 orchestration system and other usability tools. A message latency experiment shows that the end-to-end message latency overhead, when enabling all ROS 2 instrumentation, is below 0.0055 ms, which we believe is suitable for production real-time systems. ROS 2 execution information obtained using ros2 tracing can be combined with trace data from the operating system, enabling a wider range of precise analyses, that help understand an application execution, to find the cause of performance bottlenecks and other issues. The source code is available at: https://gitlab.com/rostracing/ros2 tracing.

Analyze, Debug, Optimize: Real-Time Tracing for Perception and Mapping Systems in ROS 2

“…It is based on LTTng [2], a tracer with low runtime overhead [1]. In previous work, we proposed ros2 tracing [5], a framework with instrumentation and tools for real-time tracing of ROS 2. Our subsequent work [6] extends that instrumentation, and proposes an analysis and visualization of the flow of messages across distributed systems.…”

“…To extract the chain of messages, the tracing tool needs to associate input subscriptions with output publishers. Those causal links can be specified in the application code using instrumentation as in [5]. In addition to user-defined annotation, [6] offers automatic causal links detection for output messages published directly inside the input subscription callbacks.…”

“…In this paper, we demonstrate how our novel tracing tools and analyses for ROS 2 [5], [6] can benefit the debugging and performance analysis of perception systems. In particular, we show how important metrics can be extracted from the tracing of a Simultaneous Localization and Mapping (SLAM) system.…”