Operating System Noise in the Linux Kernel

Abstract

As modern network infrastructure moves from hardware-based to software-based using Network Function Virtualization, a new set of requirements is raised for operating system developers. By using the real-time kernel options and advanced CPU isolation features common to the HPC use-cases, Linux is becoming a central building block for this new architecture that aims to enable a new set of low latency networked services. Tuning Linux for these applications is not an easy task, as it requires a deep understanding of the Linux execution model and the mix of user-space tooling and tracing features. This paper discusses the internal aspects of Linux that influence the Operating System Noise from a timing perspective. It also presents Linux's osnoise tracer, an in-kernel tracer that enables the measurement of the Operating System Noise as observed by a workload, and the tracing of the sources of the noise, in an integrated manner, facilitating the analysis and debugging of the system. Finally, this paper presents a series of experiments demonstrating both Linux's ability to deliver low OS noise (in the single-digit µs order), and the ability of the proposed tool to provide precise information about root-cause of timing-related OS noise problems.

Multi-criteria Optimization of Real-time DAGs on Heterogeneous Platforms under P-EDF

“…RTDAG provides the option to precisely calibrate the value of 𝑡 𝑡𝑖𝑐𝑘 when deployed on a new platform, by repeatedly executing the workload of a single tick and outputting a number of statistics on the measured execution times. However, any estimate on the execution time of a real-time task (synthetic or not) running on Linux under SCHED_DEADLINE can be influenced by several factors, including: interfering activities generally known as "OS noise" [32], e.g., due to the execution of IRQs or non-preemptibility sections of the kernel [33]; other tasks that may be scheduled by a scheduler that has a higher priority than SCHED_DEADLINE 6 ; cache-level interference due to other tasks (even lower-priority ones), running on the same or different cores causing an unforeseen volume of evicted L2/LLC CPU cache lines 7 . All of these factors may impact the execution times of tasks, that might sporadically happen to be longer than experienced in previous profiling campaigns.…”