Synchronization for fast and reentrant operating system kernel tracing

Abstract

To effectively trace an operating system, a performance monitoring and debugging infrastructure needs the ability to trace various execution contexts. These contexts range from kernel running as a thread to Non‐Maskable Interrupt (NMI) contexts. Given that any part of the kernel infrastructure used by a kernel tracer could lead to infinite recursion if traced, and because most kernel primitives require synchronization unsuitable for some execution contexts, all interactions of the tracing code with the existing kernel infrastructure must be considered in order to correctly inter‐operate with the existing operating system kernel. This paper presents a new low overhead tracing mechanism and motivates the choice of synchronization sequences suitable for operating system kernel tracing, namely local atomic instructions as main buffer synchronization primitive and the Read–Copy Update (RCU) mechanism to control tracing. It also proposes a wait‐free algorithm extending the time‐base needed by the tracer to 64‐bit on architectures that lack hardware 64‐bit time‐base support. Copyright © 2010 John Wiley & Sons, Ltd.

Software feature location in practice: debugging aircraft simulation systems

“…To address these concerns, we reviewed existing work in software tracing that provide practical solutions to these issues. For example, tracing tools such as LTTng [4] can instrument the system with very low overhead and almost no disturbance. Our previous work on trace abstraction, implemented in industrial tools, is an example of how traces can be simplified by reducing their size [8,9].…”

Improving J9 virtual machine with LTTng for efficient and effective tracing

“…LTTng achieves these by mainly minimizing the amount of required synchronization in trace data extraction while improving the tracing throughput. To this end, LTTng tracing is implemented on the basis of a lock‐free multi‐core high‐throughput buffer scheme, which adopts a Read‐Copy‐Update synchronization technique to obtain very efficient and wait‐free read sites at both the kernel and user‐space levels . Moreover, to further improve the throughput, a common trace format (CTF), a self‐describing highly optimized binary format, is designed to record trace events with variable sizes in a compact format.…”