The Power of Slightly More than One Sample in Randomized Load Balancing

Abstract

In many computing and networking applications, arriving tasks have to be routed to one of many servers, with the goal of minimizing queueing delays. When the number of processors is very large, a popular routing algorithm works as follows: select two servers at random and route an arriving task to the least loaded of the two. It is well known that this algorithm dramatically reduces queueing delays compared to an algorithm, which routes to a single randomly selected server. In recent cloud computing applications, it has been observed that even sampling two queues per arriving task can be expensive and can even increase delays due to messaging overhead. So there is an interest in reducing the number of sampled queues per arriving task. In this paper, we show that the number of sampled queues can be dramatically reduced by using the fact that tasks arrive in batches (called jobs). In particular, we sample a subset of the queues such that the size of the subset is slightly larger than the batch size (thus, on average, we only sample slightly more than one queue per task). Once a random subset of the queues is sampled, we propose a new load-balancing method called batch-filling to attempt to equalize the load among the sampled servers. We show that, asymptotically, our algorithm dramatically reduces the sample complexity compared to previously proposed algorithms.

The power of slightly more than one sample in randomized load balancing

“…It can be easily verified that Q (n) (t) is an irreducible and aperiodic Markov chain, and using the standard Foster-Lyapunov theorem (see, for example, [13]) it can be verified that the Markov chain is positive recurrent and hence, has a unique stationary distribution. The proof is presented in the technical report [14].…”

“…According to the lemmas in the technical report [14], we have that there existsn such that for any n n, , which converges to zero as n ! 1 since m = ⇥(log n).…”

“…Since F(s) is Lipschitz as shown in the technical report [14], from Gronwall's inequality [16], it follows that …”

Redundancy-d: The Power of d Choices for Redundancy

“…We consider the system in the limit as the number of servers k approaches infinity. We make the further assumption that in this asymptotic regime, the work in different queues is independent; such independence has been shown to hold under related policies, for example, Join-the-Shortest-Queue (JSQ) dispatching (Mitzenmacher 2001, Vvedenskaya et al 1996, Ying et al 2015. Under these assumptions, we formulate a system of differential equations that describes the evolution of the system.…”

Power-of-d-Choices with Memory: Fluid Limit and Optimality

“…In the former, the dispatcher initiates the communication fetching the requested information from the servers, while in the latter servers periodically send state information to the dispatcher. This type of model finds applications in computer and communication systems, hospitals and road networks, and there exists a significant and growing number of references; see, e.g., the recent works Ying et al [25], Gardner et al [9], Gamarnik et al [8], Gupta and Walton [11] and the references therein. Nevertheless, it is often difficult to establish whether an algorithm is better than another because in general the answer strongly depends on the underlying architecture, application or traffic conditions.…”