Adrenaline: Pinpointing and reining in tail queries with quick voltage boosting

Abstract

Reducing the long tail of the query latency distribution in modern warehouse scale computers is critical for improving performance and quality of service of workloads such as Web Search and Memcached. Traditional turbo boost increases a processor's voltage and frequency during a coarsegrain sliding window, boosting all queries that are processed during that window. However, the inability of such a technique to pinpoint tail queries for boosting limits its tail reduction benefit.In this work, we propose Adrenaline, an approach to leverage finer granularity, 10's of nanoseconds, voltage boosting to effectively rein in the tail latency with query-level precision. Two key insights underlie this work. First, emerging finer granularity voltage/frequency boosting is an enabling mechanism for intelligent allocation of the power budget to precisely boost only the queries that contribute to the tail latency; and second, per-query characteristics can be used to design indicators for proactively pinpointing these queries, triggering boosting accordingly. Based on these insights, Adrenaline effectively pinpoints and boosts queries that are likely to increase the tail distribution and can reap more benefit from the voltage/frequency boost. By evaluating under various workload configurations, we demonstrate the effectiveness of our methodology. We achieve up to a 2.50x tail latency improvement for Memcached and up to a 3.03x for Web Search over coarse-grained DVFS given a fixed boosting power budget. When optimizing for energy reduction, Adrenaline achieves up to a 1.81x improvement for Memcached and up to a 1.99x for Web Search over coarse-grained DVFS.

Tailbench: a benchmark suite and evaluation methodology for latency-critical applications

“…Many of these studies use workloads internal to datacenter operators like Google or Facebook [32,33,36,38,55,56]. Academic studies use one or a few latencycritical benchmarks [25,48,54], which limits the range of behaviors and performance requirements across which their proposed techniques can be evaluated. Some work uses more readily-available sequential and parallel batch workloads (e.g., from SPEC CPU2006 or PARSEC) and treats them as latencycritical applications [15,57].…”

“…These techniques include new cluster managers that schedule and migrate applications across systems to reduce interference [18,32,36,54], fast dynamic voltage-frequency scaling (DVFS) techniques to improve power efficiency [25,29,32,48], hardware and software schemes to use low power idle states [37,39,53], and hardware resource partitioning schemes that allow batch workloads to run alongside latency-critical ones, improving utilization [29,30,33,57].…”

“…Latency-critical applications have a wide variety of latency requirements and microarchitectural characteristics. However, most recent work in this area uses one or a few latencycritical applications in their evaluations [25,32,33,48], which do not stress a wide range of behaviors. Some prior work in this area even uses more readily-available sequential and parallel batch workloads (e.g., from SPEC CPU2006 or PAR-SEC [12]) and treats them as latency-critical applications [15,57].…”

Energy proportionality and workload consolidation for latency-critical applications

“…We observe that the impact of DVFS-only controls differs noticeably between Linux and IX: with Linux, the DVFSonly alternate frontier is very close to the Pareto frontier, meaning that a DVFS-only approach such as Pegasus [29] or Adrenaline [15] would be adequate. This is due to Linux's idling behavior, which saves resources.…”

Rubik

“…The key challenge is coping with the inherent short-term variability of latency-critical applications: requests arrive at unpredictable times and are often bursty, causing short-term spikes and queuing delays that dominate tail latency [22,25]; and the amount of work per request often varies by an order of magnitude or more [16,25].…”