Rubik

Abstract

Latency-critical workloads (e.g., web search), common in datacenters, require stable tail (e.g., 95th percentile) latencies of a few milliseconds. Servers running these workloads are kept lightly loaded to meet these stringent latency targets. This low utilization wastes billions of dollars in energy and equipment annually.Applying dynamic power management to latency-critical workloads is challenging. The fundamental issue is coping with their inherent short-term variability: requests arrive at unpredictable times and have variable lengths. Without knowledge of the future, prior techniques either adapt slowly and conservatively or rely on application-specific heuristics to maintain tail latency.We propose Rubik, a fine-grain DVFS scheme for latency-critical workloads. Rubik copes with variability through a novel, general, and efficient statistical performance model. This model allows Rubik to adjust frequencies at sub-millisecond granularity to save power while meeting the target tail latency. Rubik saves up to 66% of core power, widely outperforms prior techniques, and requires no application-specific tuning.Beyond saving core power, Rubik robustly adapts to sudden changes in load and system performance. We use this capability to design RubikColoc, a colocation scheme that uses Rubik to allow batch and latencycritical work to share hardware resources more aggressively than prior techniques. RubikColoc reduces datacenter power by up to 31% while using 41% fewer servers than a datacenter that segregates latency-critical and batch work, and achieves 100% core utilization.

Hipster: Hybrid Task Manager for Latency-Critical Cloud Workloads

“…Many important cloud workloads are latency-critical, and they require strict levels of quality-of-service (QoS) to meet user expectations [15][16][17]. A web-search, for example, must complete within a fraction of a second [18], otherwise users are likely to give up and leave.…”

“…Recent works [15][16][17][21][22][23] have shown that traditional power management practices and CPU utilization measures are unsuitable to drive task management for data center workloads. This is because prior schemes (like OS-level DVFS) work well to deliver long-term performance for batch workloads, but they can severely hurt the QoS of latency-critical data center workloads.…”

The Hipster Approach for Improving Cloud System Efficiency

“…Both these forms present a challenge for a heuristic based approach as it jumps across multiple con gurations to meet the QoS target, thereby leading to QoS violations due to rampant core oscillations. Also, the authors of Rubik [36] note that core-transitions are far more costly relative to DVFS changes.…”

“…HipsterIn performs well because it moves directly to the appropriate core con guration for a given load that satis es QoS. In addition to switching between a combination of di erent cores, HipsterIn also explores more ne-grained DVFS adaptations, which has lower overheads (of microseconds) compared with migrations between cores (order of milliseconds) [36].…”

“…However, this work explores modern Intel architectures with cache allocation technology (CAT) and DRAM bandwidth monitor, which are available from Broadwell processors released after 2015. Pegasus [40] achieves high CPU energy proportionality for low latency workloads using ne-grained DVFS techniques. TimeTrader [66] and Rubik [36] exploit request queuing latency variation and apply any available slack from queuing delay to throughput-oriented workloads to improve energy e ciency. Quasar [19] use runtime classi cation to predict interference and collocate workloads to minimize interference.…”