Abstract:We consider a class of queueing systems that consist of server pools in parallel and multiple customer classes. Customer service times are assumed to be exponentially distributed. We study the asymptotic behavior of these queueing systems in a heavy traffic regime that is known as the Halfin and Whitt many-server asymptotic regime. Our main contribution is a general framework for establishing state space collapse results in this regime for parallel server systems. In our work, state space collapse refers to a decrease in the dimension of the processes tracking the number of customers in each class waiting for service and the number of customers in each class being served by various server pools. We define and introduce a "state space collapse" function, which governs the exact details of the state space collapse. We show that a state space collapse result holds in many-server heavy traffic if a corresponding deterministic hydrodynamic model satisfies a similar state space collapse condition. Our methodology is similar in spirit to that in Bramson [10], which focuses on the conventional heavy traffic regime. We illustrate the applications of our results by establishing state space collapse results in many-server diffusion limits of static-buffer-priority V-parallel server systems, N-model parallel server systems, and minimum-expecteddelay-faster-server-first distributed server pools systems. We show for these systems that the condition on the hydrodynamic model can easily be checked using the standard tools for fluid models.

This paper studies many-server limits for multi-server queues that have a phase-type service time distribution and allow for customer abandonment. The first set of limit theorems is for critically loaded G/Ph/n + GI queues, where the patience times are independent and identically distributed following a general distribution. The next limit theorem is for overloaded G/Ph/n + M queues, where the patience time distribution is restricted to be exponential. We prove that a pair of diffusion-scaled total-customer-count and serverallocation processes, properly centered, converges in distribution to a continuous Markov process as the number of servers n goes to infinity. In the overloaded case, the limit is a multi-dimensional diffusion process, and in the critically loaded case, the limit is a simple transformation of a diffusion process. When the queues are critically loaded, our diffusion limit generalizes the result by Puhalskii and Reiman (2000) for GI /Ph/n queues without customer abandonment. When the queues are overloaded, the diffusion limit provides a refinement to a fluid limit and it generalizes a result by Whitt (2004) for M/M/n/ + M queues with an exponential service time distribution. The proof techniques employed in this paper are innovative. First, a perturbed system is shown to be equivalent to the original system. Next, two maps are employed in both fluid and diffusion scalings. These maps allow one to prove the limit theorems by applying the standard continuous-mapping theorem and the standard random-time-change theorem.

We consider a class of parallel server systems that are known as N-systems. In an N-system, there are two customer classes that are catered by servers in two pools. Servers in one of the pools are cross-trained and can serve customers from both classes, whereas all of the servers in the other pool can serve only one of the customer classes. A customer reneges from his queue if his waiting time in the queue exceeds his patience. Our objective is to minimize the total cost that includes a linear holding cost and a reneging cost. We prove that, when the service speed is pool dependent, but not class dependent, a cμ-type greedy policy is asymptotically optimal in many-server heavy traffic.

We consider customer service chat (CSC) systems where customers can receive real time service from agents using an instant messaging (IM) application over the Internet. A unique feature of these systems is that agents can serve multiple customers simultaneously. The number of customers that an agent is serving determines the rate at which each customer assigned to that agent receives service. We consider the staffing problem in CSC systems with impatient customers where the objective is to minimize the number of agents while providing a certain service level. The service level is measured in terms of the proportion of customers who abandon the system in the long run. First we propose effective routing policies based on a static planning LP, both for the cases when the arrival rate is observable and for when the rate is unobservable. We show that these routing policies minimize the proportion of abandoning customers in the long run asymptotically for large systems. We also prove that the staffing solution obtained from a staffing LP, when used with the proposed routing policies, is asymptotically optimal. We illustrate the effectiveness of our solution procedure in systems with small to large sizes via numerical and simulation experiments.Subject classifications: call center management; queuing theory and stochastic methods. Area of review: Stochastic Models.

We consider a distributed parallel server system that consists of multiple server pools and a single customer class. We show that the minimum-expected-delay faster-server-first (MED-FSF) routing policy asymptotically minimizes the stationary distribution of the total queue length and the stationary delay probability in the Halfin and Whitt regime. We propose the minimum-expected-delay load-balancing (MED-LB) routing policy to balance the utilizations of all the servers in a distributed system with no unnecessary idling. We show that this policy balances both the long-run and finite-time average utilizations over all the server pools in the Halfin and Whitt regime. We next show that, under either the MED-FSF or the MED-LB policy, a distributed system performs as well as the corresponding inverted V-system. Finally, we show that, operating under the MED-LB policy, both the distributed system and the inverted V-system have similar performances to a corresponding M/M/n system. We illustrate the quality of our asymptotic results for several parallel server systems via simulation experiments.