The power of two choices in randomized load balancing

Abstract

AbstractÐWe consider the following natural model: Customers arrive as a Poisson stream of rate !n, !`I, at a collection of n servers. Each customer chooses some constant d servers independently and uniformly at random from the n servers and waits for service at the one with the fewest customers. Customers are served according to the first-in first-out (FIFO) protocol and the service time for a customer is exponentially distributed with mean 1. We call this problem the supermarket model. We wish to know how the system behaves and in particular we are interested in the effect that the parameter d has on expected time a customer spends in the system in equilibrium. Our approach uses a limiting, deterministic model representing the behavior as n 3 I to approximate the behavior of finite systems. The analysis of the deterministic model is interesting in its own right. Along with a theoretical justification of this approach, we provide simulations that demonstrate that the method accurately predicts system behavior, even for relatively small systems. Our analysis provides surprising implications: Having d P choices leads to exponential improvements in the expected time a customer spends in the system over d I, whereas having d Q choices is only a constant factor better than d P. We discuss the possible implications for system design.

Stochastic bounds for randomized load balancing

“…However, the bounds are numerically tractable, and are remarkably tight. Moreover, the bounds illustrate that the asymptotic result from [5] can be largely inaccurate (e.g., smaller than the lower bound) in regimes with small number of servers.…”

“…A fundamental qualitative result is that SQ(2) yields an exponential improvement over SQ (1) in terms of delay, yet with a conceivably small overhead cost. This result is known as the 'power-of-two' result [5].…”

“…An exact solution on the delay distribution was obtained in an asymptotic regime in the total number of servers, i.e., for N → ∞ [5]; this solution was instrumental to showing the 'power-of-two' result. Lower and upper bounds on delay were obtained for the particular case when d = N , i.e., JSQ.…”

“…In order to reduce the feedback cost, and yet to keep the delay 'small', JSQ has been generalized to SQ(d), whereby the dispatcher runs JSQ only for a subset of d randomly sampled servers from the uniform distribution (see Mitzenmacher [5] and Luczak and McDiarid [3]). Note that SQ(d) reduces to a simple uniform random selection when d = 1, and to JSQ when d = N , where N is the total number of servers.…”

Workload Modeling for Computer Systems Performance Evaluation

“…The first question is whether there is enough of a load imbalance to justify active migration of processes to improve the balance. In particular, an underloaded machine and an overloaded machine need to be paired up, which requires machines to know about each other [52,495].…”