Energy proportionality and workload consolidation for latency-critical applications

Abstract

Energy proportionality and workload consolidation are important objectives towards increasing efficiency in largescale datacenters. Our work focuses on achieving these goals in the presence of applications with µs-scale tail latency requirements. Such applications represent a growing subset of datacenter workloads and are typically deployed on dedicated servers, which is the simplest way to ensure low tail latency across all loads. Unfortunately, it also leads to low energy efficiency and low resource utilization during the frequent periods of medium or low load.We present the OS mechanisms and dynamic control needed to adjust core allocation and voltage/frequency settings based on the measured delays for latency-critical workloads. This allows for energy proportionality and frees the maximum amount of resources per server for other background applications, while respecting service-level objectives. Monitoring hardware queue depths allows us to detect increases in queuing latencies. Carefully coordinated adjustments to the NIC's packet redirection table enable us to reassign flow groups between the threads of a latency-critical application in milliseconds without dropping or reordering packets. We compare the efficiency of our solution to the Pareto-optimal frontier of 224 distinct static configurations. Dynamic resource control saves 44%-54% of processor energy, which corresponds to 85%-93% of the Pareto-optimal upper bound. Dynamic resource control also allows background jobs to run at 32%-46% of their standalone throughput, which corresponds to 82%-92% of the Pareto bound.

The Hipster Approach for Improving Cloud System Efficiency

“…Previous studies [6,19,56,57] have shown that data centers like Google and Facebook operate at peak load only for a small fraction of the day, where the power consumption and load are proportional. The number of queries per second during a web search, a typical load at Google data center, varies between about 5% and 80% of maximum capacity [31,40,47].…”

Analysis of power consumption in heterogeneous virtual machine environments

“…In this way is achieved the reduction of the power consumption. Usually, the workload placement problem is modeled as a multi-dimensional bin-packaging problem, as expressed in [45], [42], [29]. Moreover, meta-heuristics such as Ant Colony Optimization [14], [15], [17], Genetic Algorithms [25], [46], [44], [24] are used for power consumption optimization.…”

Peak efficiency aware scheduling for highly energy proportional servers

“…Energy proportional computing: Energy proportional computing have been a major area of research [1], [5], [6], [9], [15], [18], [35]- [37]. In addition, measurements, metrics and trends of energy proportionality have all been well studied [19], [21], [22], [35], [38].…”

“…In addition, measurements, metrics and trends of energy proportionality have all been well studied [19], [21], [22], [35], [38]. Energy proportionality techniques has been proposed at the cluster level through workload consolidation and dynamic capacity management with the goal of "right-sizing" the amount of servers in the data center [10], [14]- [16], [36], [37], [39]. These migrationbased techniques are best suited for coarse-grain workload fluctuations in the order of minutes -hours, and typically assumes a stable power budget.…”