Universality of load balancing schemes on the diffusion scale

Abstract

We consider a system of N parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among d randomly selected servers otherwise (1 ≤ d ≤ N ). This load balancing scheme subsumes the so-called Join-the-Idle Queue (JIQ) policy (d = 1) and the celebrated Join-the-Shortest Queue (JSQ) policy (d = N ) as two crucial special cases. We develop a stochastic coupling construction to obtain the diffusion limit of the queue process in the Halfin-Whitt heavy-traffic regime, and establish that it does not depend on the value of d, implying that assigning tasks to idle servers is sufficient for diffusion level optimality. * Corresponding author: d.mukherjee@tue.nl Load balancing schemes can be broadly categorized as static (open-loop), dynamic (closed-loop), or some intermediate blend, depending on the amount of real-time feedback or state information (e.g. queue lengths or load measurements) that is used in assigning tasks. Within the category of dynamic policies, one can further distinguish between push-based and pull-based approaches, depending on whether the initiative resides with a dispatcher actively collecting feedback from the servers, or with the servers advertizing their availability or load status. The use of state information naturally allows dynamic policies to achieve better performance and greater resource pooling gains, but also involves higher implementation complexity and potentially substantial communication overhead. The latter issue is particularly pertinent in large-scale data centers, which deploy thousands of servers and handle massive demands, with service requests coming in at huge rates.In the present paper we focus on a basic scenario of N parallel queues with identical servers, exponentially distributed service requirements, and a service discipline at each individual server that is oblivious to the actual service requirements (e.g. FCFS). In this canonical case, the so-called Join-the-Shortest-Queue (JSQ) policy has several strong optimality properties, and in particular minimizes the overall mean delay among the class of non-anticipating load balancing policies that do not have any advance knowledge of the service requirements [3,16,18]. (Relaxing any of the three above-mentioned assumptions tends to break the optimality properties of the JSQ policy, and renders the delay-minimizing policy quite complex or even counter-intuitive, see for instance [5,7,17].)In order to implement the JSQ policy, a dispatcher requires instantaneous knowledge of the queue lengths at all the servers, which may give rise to a substantial communication burden, and may not be scalable in scenarios with large numbers of servers. The latter issue has motivated consideration of so-called JSQ(d) policies, where the dispatcher assigns an incoming task to a server with the shortest queue among d servers selected uniformly at random. Me...

Analysis of the shortest relay queue policy in a cooperative random access network with collisions

“…Such schemes improve the system's performance by dividing the work effectively among the participating nodes. Under certain conditions (identical servers, exponential service times, and service protocols that do not depend on the service requirement, such as the FCFS), the so-called join the shortest queue (JSQ) policy has several strong optimality properties: The JSQ policy minimises the overall mean delay among the class of load balancing policies that do not have any advance knowledge of the service requirements [19,40,60].…”

“…For |r| < 1, Φ 0 (r) is regular, and E(r Q2 2 (Q 1 = 0)) is regular for |r 2 | < 1. Thus, the right-hand side in (40) can be continued analytically into |r 2 | < 1, Re(1 − 1 r2 ) ≤ 0. Since Φ 0 (r) is well defined for |r| < 1 and Re(1 − 1 r ) ≤ 0, we conclude that Φ 0 (δ 1 (r 2 )r 2 ) can be analytically continued into 0)) is real for r 2 ∈ G 1 , Equation (40) yields for r 2 ∈ G 1 ,…”

“…The employed cooperation strategy among source and relays is queue-based, with as ultimate goal to minimise the total transmission time, i.e., the time that is needed to transmit a packet from the source to the destination. To this end, we consider join the shortest queue policy as it seems to be the most appropriate for such a wireless network [39,40,60,63].…”

Optimal Service Elasticity in Large-Scale Distributed Systems

“…Results in [25] indicate that the JIQ policy exhibits the same diffusion-limit behavior as the Join-the-Shortest-Queue (JSQ) strategy, and thus achieves optimality at the diffusion level. These results show that the JIQ policy provides asymptotically optimal delay performance while only involving minimal communication overhead (at most one message per task).…”

Hyper-Scalable JSQ with Sparse Feedback

“…Many popular schemes have also been analyzed in the Halfin-Whitt heavy-traffic regime [18] and in the nondegenerate slowdown regime [10], in which the JIQ scheme is not necessarily optimal.…”