Optimal Service Elasticity in Large-Scale Distributed Systems

Abstract

A fundamental challenge in large-scale cloud networks and data centers is to achieve highly efficient server utilization and limit energy consumption, while providing excellent userperceived performance in the presence of uncertain and timevarying demand patterns. Auto-scaling provides a popular paradigm for automatically adjusting service capacity in response to demand while meeting performance targets, and queue-driven auto-scaling techniques have been widely investigated in the literature. In typical data center architectures and cloud environments however, no centralized queue is maintained, and load balancing algorithms immediately distribute incoming tasks among parallel queues. In these distributed settings with vast numbers of servers, centralized queue-driven auto-scaling techniques involve a substantial communication overhead and major implementation burden, or may not even be viable at all.Motivated by the above issues, we propose a joint autoscaling and load balancing scheme which does not require any global queue length information or explicit knowledge of system parameters, and yet provides provably near-optimal service elasticity. We establish the fluid-level dynamics for the proposed scheme in a regime where the total traffic volume and nominal service capacity grow large in proportion. The fluid-limit results show that the proposed scheme achieves asymptotic optimality in terms of user-perceived delay performance as well as energy consumption. Specifically, we prove that both the waiting time of tasks and the relative energy portion consumed by idle servers vanish in the limit. At the same time, the proposed scheme operates in a distributed fashion and involves only constant communication overhead per task, thus ensuring scalability in massive data center operations. Extensive simulation experiments corroborate the fluid-limit results, and demonstrate that the proposed scheme can match the user performance and energy consumption of state-of-the-art approaches that do take full advantage of a centralized queue.

CloudCast: Characterizing Public Clouds Connectivity

“…Broader aspects of large data center and cloud optimization have been pushed to the forefront ever since the appearance of public clouds, captivating both the research community and the industry. Challenges such as auto-scaling [24,27], job scheduling [31], energy consumption [37] to name a few are interleaved with budgeting issues [13] and other concerns. When it comes to service pricing, as mentioned previously, cloud providers do not offer at this time quality of service guarantees in terms of data delivery.…”

Macroscopic modelling and analysis of flows during rush-hour congestion

“…The process Q(t) is a timeinhomogenous density dependent population process; see e.g. [27,28,29] for recent queueing theoretic applications of such processes. We can study the fluid limit of the system, that is, we consider a sequence of models Q K (t), the Kth model having arrival rate λ f (t)K for the f th flow and departure rate K μn (x/K) from the nth neighbourhood when there are x vehicles in that neighbourhood.…”

A Framework for Allocating Server Time to Spot and On-Demand Services in Cloud Computing

“…The PTC policy is more advanced but recently has been applied to the cluster management practice [27]; it can achieve a higher utilization of servers [24]. More discussion on dispatching policies can be found in [14,15,18,25]. Once the job j is dispatched to a server, the server will be occupied by the on-demand job owner during the period [a j , d j ], i.e., from the beginning of slot a j until the end of slot d j .…”