Bubble-flux

Abstract

Ensuring the quality of service (QoS) for latency-sensitive applications while allowing co-locations of multiple applications on servers is critical for improving server utilization and reducing cost in modern warehouse-scale computers (WSCs). Recent work relies on static profiling to precisely predict the QoS degradation that results from performance interference among co-running applications to increase the number of "safe" co-locations. However, these static profiling techniques have several critical limitations: 1) a priori knowledge of all workloads is required for profiling, 2) it is difficult for the prediction to capture or adapt to phase or load changes of applications, and 3) the prediction technique is limited to only two co-running applications.To address all of these limitations, we present Bubble-Flux, an integrated dynamic interference measurement and online QoS management mechanism to provide accurate QoS control and maximize server utilization. Bubble-Flux uses a Dynamic Bubble to probe servers in real time to measure the instantaneous pressure on the shared hardware resources and precisely predict how the QoS of a latency-sensitive job will be affected by potential co-runners. Once "safe" batch jobs are selected and mapped to a server, Bubble-Flux uses an Online Flux Engine to continuously monitor the QoS of the latency-sensitive application and control the execution of batch jobs to adapt to dynamic input, phase, and load changes to deliver satisfactory QoS. Batch applications remain in a state of flux throughout execution. Our results show that the utilization improvement achieved by Bubble-Flux is up to 2.2x better than the prior static approach.

Seer

“…Apart from software heterogeneity, datacenter hardware is also becoming increasingly heterogeneous as special-purpose architectures [20-22, 39, 55] and FPGAs are used to accelerate critical operations [19,25,40,75]. This adds to the existing server heterogeneity in the cloud where servers are progressively replaced and upgraded over the datacenter's provisioned lifetime [31,33,65,68,95], and further complicates the effort to guarantee predictable performance.…”

“…A second line of work tries to identify resources that will allow a new, potentially-unknown application to meet its performance (throughput or tail latency) requirements [29,31,32,34,66,68,95]. Paragon uses classification to determine the impact of platform heterogeneity and workload interference on an unknown, incoming workload [30,31].…”

“…Nathuji et al developed a feedback-based scheme that tunes resource assignments to mitigate memory interference [69]. Yang et al developed an online scheme that detects memory pressure and finds colocations that avoid interference on latency-sensitive workloads [95]. Similarly, DeepDive detects and manages interference between co-scheduled workloads in a VM environment [71].…”

Performance & Energy Tradeoffs for Dependent Distributed Applications Under System-wide Power Caps

“…Again, this work can be done at the processor-level [23,24,44,56], DRAM [11], storage [30], and across a data-center [33,39]. At the data-center or cluster-level, power can be saved by consolidating workloads to use fewer physical machines [14,27,34,35,37,54], coordinating co-existing applications [45], and scheduling with green power [20] Scheduling jobs under a power cap has recently become a major concern for HPC operating systems [7,15] and job schedulers [3,19]. Recent work suggests that HPC workloads can actually achieve higher performance by over-provisioning large-scale installationssuch that using all nodes at full capacity would drastically violate the power budget-and severely power capping the individual nodes [47].…”

The IX Operating System

“…Our approach explores this issue at higher throughputs and with tighter-latency SLOs. Bubble-Flux [Yang et al 2013] additionally controls background threads; we control background and latency-sensitive threads. CPI 2 [Zhang et al 2013] detects performance interference by observing changes in CPI and throttles offending jobs.…”