An analysis of performance evolution of Linux's core operations

Abstract

This paper presents an analysis of how Linux's performance has evolved over the past seven years. Unlike recent works that focus on OS performance in terms of scalability or service of a particular workload, this study goes back to basics: the latency of core kernel operations (e.g., system calls, context switching, etc.). To our surprise, the study shows that the performance of many core operations has worsened or fluctuated significantly over the years. For example, the select system call is 100% slower than it was just two years ago. An in-depth analysis shows that over the past seven years, core kernel subsystems have been forced to accommodate an increasing number of security enhancements and new features. These additions steadily add overhead to core kernel operations but also frequently introduce extreme slowdowns of more than 100%. In addition, simple misconfigurations have also severely impacted kernel performance. Overall, we find most of the slowdowns can be attributed to 11 changes. Some forms of slowdown are avoidable with more proactive engineering. We show that it is possible to patch two security enhancements (from the 11 changes) to eliminate most of their overheads. In fact, several features have been introduced to the kernel unoptimized or insufficiently tested and then improved or disabled long after their release. Our findings also highlight both the feasibility and importance for Linux users to actively configure their systems to achieve an optimal balance between performance, functionality, and security: we discover that 8 out of the 11 changes can be avoided by reconfiguring the kernel, and the other 3 can be disabled through simple patches. By disabling the 11 changes with the goal of optimizing performance, we speed up Redis, Apache, and Nginx benchmark workloads by as much as 56%, 33%, and 34%, respectively.

A Linux in unikernel clothing

“…A general kernel (lupine-general) that supports 20 applications in Section 3 does not sacrifice application performance. We note that, as a unikernel-like system with a single trust domain, Lupine does not require the use of many recent security enhancements that have been shown to incur significant slowdowns, oftentimes more than 100% [52]. We attribute much of Lupine's 20% (or greater) application performance improvement (when compared to baseline) to disabling these enhancements.…”

Analyzing the Overhead of File Protection by Linux Security Modules

“…In this paper, this method is followed. Previous standard filesystem benchmarks are using Intel lkp-tests suite [5] and previous papers [24,31,42]: (1) filebench [21], (2) lmbench (2.5 and [22] (3) FS-Mark [29] and (4) unix-bench [39]. lmbench3 [22] adds scalability test to lm-bench2.5 [22], however it misses chmod/rename etc., which are essential for security performance tests.…”

Energy and run-time performance practices in software engineering

“…According to the related work [145], the main reason of performance decline caused by the CPU vulnerability patches is the flushing of buffers after context-switching. Specifically, the flushing occurs in order to clean stale data from buffers to avoid data leakage through sidechannel attacks.…”

“…The Meltdown vulnerability patch is flushing the Translation Lookaside Buffers (TLBs) for each switching between the kernel-and user-space. Note that flushing TLBs, increases the number of TLB load misses [145,28]. This, in turn, leads to a severe performance degradation due to expensive memory accesses, which result in an additional energy and run-time performance overhead.…”

“…Flushing out BTBs reduces drastically the prediction rate of branches. Therefore, the CPU is forced to discard data for the associated mispredictions and waste resources executing unneeded instructions [145,28]. In a similar manner, MDS vulnerability patch flushes out stale data from different CPU buffers (i.e., line-fill buffers, store buffers, and load ports) that may have been loaded from previous operations [28].…”