TimeTrader

Abstract

Online Search (OLS) is a key component of many popular Internet services. Datacenters running OLS consume significant amounts of energy. However, reducing their energy is challenging due to their tight response time requirements. A key aspect of OLS is that each user query goes to all or many of the nodes in the cluster, so that the overall time budget is dictated by the tail of the replies' latency distribution; replies see latency variations both in the network and compute. Previous work proposes to achieve load-proportional energy by slowing down the computation at lower datacenter loads based directly on response times (i.e., at lower loads, the proposal exploits the average slack in the time budget provisioned for the peak load). In contrast, we propose TimeTrader to reduce energy by exploiting the latency slack in the sub-critical replies which arrive before the deadline (e.g., 80% of replies are 3-4x faster than the tail). This slack is present at all loads and subsumes the previous work's load-related slack. While the previous work shifts the leaves' response time distribution to consume the slack at lower loads, TimeTrader reshapes the distribution at all loads by slowing down individual sub-critical nodes without increasing missed deadlines. TimeTrader exploits slack in both the network and compute budgets. Further, TimeTrader leverages Earliest Deadline First scheduling to largely decouple critical requests from the queuing delays of subcritical requests which can then be slowed down without hurting critical requests. A combination of real-system measurements and at-scale simulations shows that without adding to missed deadlines, TimeTrader saves 15% and 40% energy at 90% and 30% loading, respectively, in a datacenter with 512 nodes, whereas previous work saves 0% and 30%. Further, as a proof-of-concept, we build a small-scale real implementation to evaluate TimeTrader and show 10-30% energy savings.

QoS for best‐effort batch jobs in container‐based cloud

“…Paragon 9 and Quasar 10 predict and detect the resource interference between jobs through workload profiling and application classification, and select the optimal node accordingly. In addition, there were studies that supported granular partitioning to analyze and eliminate the cases of interference between latency‐sensitive jobs and best‐effort batch jobs when deploying the application to the node 11‐16 . These studies focused on resource guarantee and efficient idle resource consumption.…”

“…In addition, there were studies that supported granular partitioning to analyze and eliminate the cases of interference between latency-sensitive jobs and best-effort batch jobs when deploying the application to the node. [11][12][13][14][15][16] These studies focused on resource guarantee and efficient idle resource consumption. On the other hand, detailed control of the resources used by best-effort batch jobs with a low priority was not studied thoroughly.…”

“…Thus, it is highly required to provide more delicate QoS control for best‐effort batch jobs. However, since the performance of high‐priority jobs is deservingly more important than that of low‐priority jobs, existing studies on QoS in the cloud have focused on resource guarantee and isolation of high‐priority jobs 6‐16 . In this paper, we aim at providing QoS for best‐effort batch jobs, while preventing side effects that may occur to high‐priority jobs.…”

“…However, since the performance of high-priority jobs is deservingly more important than that of low-priority jobs, existing studies on QoS in the cloud have focused on resource guarantee and isolation of high-priority jobs. [6][7][8][9][10][11][12][13][14][15][16] In this paper, we aim at providing QoS for best-effort batch jobs, while preventing side effects that may occur to high-priority jobs. We particularly target the container-based cloud.…”

Energy optimising methodologies on heterogeneous data centres

“…Recent works [24,40,60,[161][162][163] have shown that traditional power management practices and CPU utilisation measures are unsuitable to drive task management for data centre workloads. This is because prior schemes (like OS-level DVFS) work well to deliver long-term performance for batch workloads, but they can severely hurt the QoS of latency-critical data centre workloads.…”