Pull-based load distribution among heterogeneous parallel servers: the case of multiple routers

Stolyar, Alexander L.

doi:10.1007/s11134-016-9508-8

Cited by 47 publications

(48 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The fluid-level optimality of the JIQ scheme was shown in [32,33]. This observation thus establishes the optimality of the fluid-limit trajectory under the TABS scheme for suitable parameter values in terms of response time performance.…”

Section: Proposition 34 (Asymptotic Optimality Of Tabs Scheme) In Amentioning

confidence: 53%

“…Fluid-limit results in [32,33] show that under Markovian assumptions, the JIQ policy achieves a zero probability of wait for any fixed subcritical load per server in a regime where the total number of servers grows large. Results in [25] indicate that the JIQ policy exhibits the same diffusion-limit behavior as the Join-the-Shortest-Queue (JSQ) strategy, and thus achieves optimality at the diffusion level.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Service Elasticity in Large-Scale Distributed Systems

Mukherjee

Dhara

Borst

et al. 2017

Proc. ACM Meas. Anal. Comput. Syst.

View full text Add to dashboard Cite

A fundamental challenge in large-scale cloud networks and data centers is to achieve highly efficient server utilization and limit energy consumption, while providing excellent userperceived performance in the presence of uncertain and timevarying demand patterns. Auto-scaling provides a popular paradigm for automatically adjusting service capacity in response to demand while meeting performance targets, and queue-driven auto-scaling techniques have been widely investigated in the literature. In typical data center architectures and cloud environments however, no centralized queue is maintained, and load balancing algorithms immediately distribute incoming tasks among parallel queues. In these distributed settings with vast numbers of servers, centralized queue-driven auto-scaling techniques involve a substantial communication overhead and major implementation burden, or may not even be viable at all.Motivated by the above issues, we propose a joint autoscaling and load balancing scheme which does not require any global queue length information or explicit knowledge of system parameters, and yet provides provably near-optimal service elasticity. We establish the fluid-level dynamics for the proposed scheme in a regime where the total traffic volume and nominal service capacity grow large in proportion. The fluid-limit results show that the proposed scheme achieves asymptotic optimality in terms of user-perceived delay performance as well as energy consumption. Specifically, we prove that both the waiting time of tasks and the relative energy portion consumed by idle servers vanish in the limit. At the same time, the proposed scheme operates in a distributed fashion and involves only constant communication overhead per task, thus ensuring scalability in massive data center operations. Extensive simulation experiments corroborate the fluid-limit results, and demonstrate that the proposed scheme can match the user performance and energy consumption of state-of-the-art approaches that do take full advantage of a centralized queue.

show abstract

Section: Proposition 34 (Asymptotic Optimality Of Tabs Scheme) In Amentioning

confidence: 53%

Section: Introductionmentioning

confidence: 99%

Optimal Service Elasticity in Large-Scale Distributed Systems

Mukherjee

Dhara

Borst

et al. 2017

Proc. ACM Meas. Anal. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…where the last inequality follows from the linearity of expectation and (33). Now, using (36) in (35) yields…”

Section: A Stochastic Updatesmentioning

confidence: 98%

LSQ: Load Balancing in Large-Scale Heterogeneous Systems With Multiple Dispatchers

Vargaftik

Keslassy

Orda

2020

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

Nowadays, the efficiency and even the feasibility of traditional load-balancing policies are challenged by the rapid growth of cloud infrastructure and the increasing levels of server heterogeneity. In such heterogeneous systems with many loadbalancers, traditional solutions, such as JSQ, incur a prohibitively large communication overhead and detrimental incast effects due to herd behavior. Alternative low-communication policies, such as JSQ(d) and the recently proposed JIQ, are either unstable or provide poor performance.We introduce the Local Shortest Queue (LSQ) family of load balancing algorithms. In these algorithms, each dispatcher maintains its own, local, and possibly outdated view of the server queue lengths, and keeps using JSQ on its local view. A small communication overhead is used infrequently to update this local view. We formally prove that as long as the error in these local estimates of the server queue lengths is bounded in expectation, the entire system is strongly stable. Finally, in simulations, we show how simple and stable LSQ policies exhibit appealing performance and significantly outperform existing low-communication policies, while using an equivalent communication budget. In particular, our simple policies often outperform even JSQ due to their reduction of herd behavior. We further show how, by relying on smart servers (i.e., advanced pull-based communication), we can further improve performance and lower communication overhead.

show abstract

“…Designed to address the lack of full queue-length information in a system with a large number of parallel queues, the PoT algorithm routes an incoming job to the shorter one between two randomly sampled queues. The same design consideration underlies pull-based variants of PoT (Badonnel and Burgess 2008, Lu et al 2011, Stolyar 2015, 2017, and the partially centralized scheduling policy by Tsitsiklis and Xu (2012) that has access to complete queue-length information only a small fraction of the time. Beyond the realm of computer networks, information constraints are also prominent in systems with humans in the loop.…”

Section: Related Literaturementioning

confidence: 99%

Information, Memory and Capacity in Dynamic Resource Allocation

Zhong

2019

Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems

View full text Add to dashboard Cite

We propose a general framework, dubbed Stochastic Processing under Imperfect Information (SPII), to study the impact of information constraints and memories on dynamic resource allocation. The framework involves a Stochastic Processing Network (SPN) scheduling problem in which the scheduler may access the system state only through a noisy channel, and resource allocation decisions must be carried out through the interaction between an encoding policy (who observes the state) and allocation policy (who chooses the allocation). Applications in the management of large-scale data centers and human-in-the-loop service systems are among our chief motivations.We quantify the degree to which information constraints reduce the size of the capacity region in general SPNs, and how such reduction depends on the amount of memories available to the encoding and allocation policies. Using a novel metric, capacity factor , our main theorem characterizes the reduction in capacity region (under "optimal" policies) for all non-degenerate channels, and across almost all combinations of memory sizes. Notably, the theorem demonstrates, in substantial generality, that p1q the presence of a noisy channel always reduces capacity, p2q more memory for the allocation policy always improves capacity, and p3q more memory for the encoding policy has little to no effect on capacity. Finally, all of our positive (achievability) results are established through constructive, implementable policies.Our proof program involves the development of a host of new techniques, largely from first principles, by combining ideas from information theory, learning and queueing theory. As a sub-module of one of the policies proposed, we create a simple yet powerful generalization of the Max-Weight policy, in which individual Markov chains are selected dynamically, in a manner analogous to how schedules are used in a conventional Max-Weight policy. 1

show abstract

Pull-based load distribution among heterogeneous parallel servers: the case of multiple routers

Cited by 47 publications

References 14 publications

Optimal Service Elasticity in Large-Scale Distributed Systems

Optimal Service Elasticity in Large-Scale Distributed Systems

LSQ: Load Balancing in Large-Scale Heterogeneous Systems With Multiple Dispatchers

Information, Memory and Capacity in Dynamic Resource Allocation

Contact Info

Product

Resources

About