Towards energy proportionality for large-scale latency-critical workloads

Abstract

Reducing the energy footprint of warehouse-scale computer (WSC) systems is key to their affordability, yet difficult to achieve in practice. The lack of energy proportionality of typical WSC hardware and the fact that important workloads (such as search) require all servers to remain up regardless of traffic intensity renders existing power management techniques ineffective at reducing WSC energy use. We present PEGASUS, a feedback-based controller that significantly improves the energy proportionality of WSC systems, as demonstrated by a real implementation in a Google search cluster. PEGASUS uses request latency statistics to dynamically adjust server power management limits in a fine-grain manner, running each server just fast enough to meet global service-level latency objectives. In large cluster experiments, PEGASUS reduces power consumption by up to 20%. We also estimate that a distributed version of PEGASUS can nearly double these savings

Tailbench: a benchmark suite and evaluation methodology for latency-critical applications

“…These workloads are architected in a high-fanout, multitiered configuration, with root nodes receiving user requests and farming them out to leaf nodes for processing. Thousands of leaf nodes may collaborate to serve each user request [9,17,32], and the latency perceived by the user is determined by the few slowest nodes, since the root node must wait for results from most or all leaf nodes to produce the final response. Thus, to ensure acceptable end-to-end latencies, the tail latencies (e.g., 95 th or 99 th percentile latencies) of leaf nodes should be small (e.g., a few milliseconds) and uniform across nodes.…”

“…These techniques include new cluster managers that schedule and migrate applications across systems to reduce interference [18,32,36,54], fast dynamic voltage-frequency scaling (DVFS) techniques to improve power efficiency [25,29,32,48], hardware and software schemes to use low power idle states [37,39,53], and hardware resource partitioning schemes that allow batch workloads to run alongside latency-critical ones, improving utilization [29,30,33,57].…”

“…Many of these studies use workloads internal to datacenter operators like Google or Facebook [32,33,36,38,55,56]. Academic studies use one or a few latencycritical benchmarks [25,48,54], which limits the range of behaviors and performance requirements across which their proposed techniques can be evaluated.…”

“…For example, web search leaf nodes must provide 99 th percentile latencies of a few milliseconds [17,49]. The need for low tail latency presents new challenges and opportunities for system designers, as many hardware and software techniques in current systems seek to improve long-term average performance, but do not help or even hurt short-term worst-case latency [28,32].…”

“…Latency-critical applications have a wide variety of latency requirements and microarchitectural characteristics. However, most recent work in this area uses one or a few latencycritical applications in their evaluations [25,32,33,48], which do not stress a wide range of behaviors. Some prior work in this area even uses more readily-available sequential and parallel batch workloads (e.g., from SPEC CPU2006 or PAR-SEC [12]) and treats them as latency-critical applications [15,57].…”