Ubik

Abstract

Chip-multiprocessors (CMPs) must often execute workload mixes with different performance requirements. On one hand, user-facing, latency-critical applications (e.g., web search) need low tail (i.e., worst-case) latencies, often in the millisecond range, and have inherently low utilization. On the other hand, compute-intensive batch applications (e.g., MapReduce) only need high long-term average performance. In current CMPs, latency-critical and batch applications cannot run concurrently due to interference on shared resources. Unfortunately, prior work on quality of service (QoS) in CMPs has focused on guaranteeing average performance, not tail latency.In this work, we analyze several latency-critical workloads, and show that guaranteeing average performance is insufficient to maintain low tail latency, because microarchitectural resources with state, such as caches or cores, exert inertia on instantaneous workload performance. Last-level caches impart the highest inertia, as workloads take tens of milliseconds to warm them up. When left unmanaged, or when managed with conventional QoS frameworks, shared last-level caches degrade tail latency significantly. Instead, we propose Ubik, a dynamic partitioning technique that predicts and exploits the transient behavior of latency-critical workloads to maintain their tail latency while maximizing the cache space available to batch applications. Using extensive simulations, we show that, while conventional QoS frameworks degrade tail latency by up to 2.3×, Ubik simultaneously maintains the tail latency of latency-critical workloads and significantly improves the performance of batch applications.

Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations

“…Improving server cost-efficiency via multi-tenancy has been studied extensively in prior literature [18], [19], [20], [21], [22], [23]. As co-located multi-tenant tasks contend for shared resources, prior work has focused on how to minimize interference and performance unpredictability.…”

Slack squeeze coded computing for adaptive straggler mitigation

“…S 2 C 2 differs from these techniques because it is a pro-active approach to straggler mitigation. In works [18,25,46] the authors explore system sources of tail latency from system and implement mechanisms to eliminate these causes. In another set of works, [9,24,31,48], authors focus on improving resource efficiency while providing low latency.…”

A Hardware Approach to Fairly Balance the Inter-Thread Interference in Shared Caches

“…They propose a QoS metric that modulates the allocated cache space for a given thread. Ubik [16] employs ZCaches [17], a radical change from traditional caches, to partition the cache. Their goal is to provide QoS while at the same time improving the performance of batch applications.…”