Scheduling a multi class queue with many exponential servers: asymptotic optimality in heavy traffic

Atar, Rami; Mandelbaum, Avishai; Reiman, Martin I.

doi:10.1214/105051604000000233

Cited by 120 publications

(166 citation statements)

References 34 publications

Supporting

Mentioning

161

Contrasting

Order By: Relevance

“…Note that (4.5) corresponds to the claim (49) in [7]. Then, we follow the method in [7,Theorem 4 (ii)] and obtain that lim…”

Section: Letmentioning

confidence: 99%

“…In this subsection, we first establish an estimate forX n by using an auxiliary process. Then, following a similar approach as in [7], we prove asymptotic optimality for the discounted problem. Given the admissible scheduling policyÛ n , letX n be a d-dimensional process defined by for i ∈ I. Lemma 4.1.…”

Section: 2mentioning

confidence: 99%

“…As a consequence, (4 Next, we show that there exists a sequence of admissible scheduling policesÛ n which attains optimality (asymptotically). Observe from [7] that the partial derivates ofV ϑ in Theorem 4.1 up to order two are locally Hölder continuous (see also [3,Lemma 3.5.4]), and the optimal valueV ϑ has polynomial growth. By [7,Theorem 1], there exists an optimal control v h ∈ U SM for the discounted problem.…”

Section: 2mentioning

confidence: 99%

“…The system operates in the "averaged" Halfin-Whitt (H-W) regime, namely, it is critically loaded in an average sense, but it may be underloaded or overloaded for a given state of the environment. This situation is different from the standard H-W regime for many-server queues, which requires that the system is critically loaded as the arrival rates and number of servers get large; see, e.g., [2,7,16,17]. policies, i.e., work-conserving and non-preemptive polices.…”

Section: Introductionmentioning

confidence: 96%

See 3 more Smart Citations

Optimal Control of Markov-Modulated Multiclass Many-Server Queues

et al. 2019

View full text Add to dashboard Cite

We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means that it is critically loaded in the average sense, although not necessarily in each state of the Markov process. We show that under any static priority policy, the Markov-modulated diffusion-scaled queueing process is geometrically ergodic. This is accomplished by employing a solution to an associated Poisson equation in order to construct a suitable Lyapunov function. We establish a functional central limit theorem for the diffusion-scaled queueing process and show that the limiting process is a controlled diffusion with piecewise linear drift and constant covariance matrix. We address the infinite-horizon discounted and long-run average (ergodic) optimal control problems and establish asymptotic optimality.We first establish a FCLT for the Markov-modulated diffusion-scaled queueing processes under any admissible scheduling policy (only considering work-conserving and preemptive policies). Proper scaling is needed in order to establish weak convergence of the queueing processes. In particular, since the arrival processes are of order n, and the switching rates of the background process are assumed to be of order n α for α > 0, the queueing processes are centered at the 'averaged' steady state, which is of order n, and are then scaled down by a factor of an n β , with β := max{ 1 /2, 1− α /2}, in the diffusion scale. Thus, when α ≥ 1, we have the usual diffusion scaling with β = 1 /2, which is due to the fact that the very fast switching of the environment results in an 'averaging' effect for the arrival, service and abandonment processes of the queueing dynamics. The limit queueing process is a piecewise Ornstein-Uhlenbeck diffusion process with a drift and covariance given by the corresponding 'averaged' quantities under the stationary distribution of the background process. When α = 1, both the variabilities of the queueing and background processes are captured in the covariance matrix, while when α > 1, only the variabilities of the queueing process is captured. On the other hand, when α < 1, the proper diffusion scaling requires β = 1 − α /2, for which we obtain a similar piecewise Ornstein-Uhlenbeck diffusion process with the covariance matrix capturing the variabilities of the background process only.The ergodic properties of this class of piecewise linear diffusions (and Lévy-driven stochastic differential equations) have been studied in [6,13], and these results can be applied directly to our model. The study of the ergodic properties of the diffusion-scaled processes, however, is challenging. Ergodicity of switching Markov processes has been an active research subject. For switching diffusions, stability has been studied in [18,19,21]. However, studies of ergodicity of switching Markov processes are scarce. Recently in [8,9], some kind of hypoellipticity criterion with H...

show abstract

“…Note that (4.5) corresponds to the claim (49) in [7]. Then, we follow the method in [7,Theorem 4 (ii)] and obtain that lim…”

Section: Letmentioning

confidence: 99%

Section: 2mentioning

confidence: 99%

Section: 2mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

See 2 more Smart Citations

Optimal Control of Markov-Modulated Multiclass Many-Server Queues

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Heavy traffic analysis can be used to deal with a class of important problems of controlled queues and networks by means of fluid and diffusion approximation. Readers may refer to, for example, Kushner [203], Kushner and Ramachandran [205], Kushner and Martins [204]; Harrison [152], Plambeck et al [257]; Chen and Yao [76], Atar et al [26].…”

Section: (9) Heavy Traffic Analysis For Controlled Queues and Networkmentioning

confidence: 99%

An Overview for Markov Decision Processes in Queues and Networks

Fan

et al. 2019

Stochastic Models in Reliability, Network Security and System Safety

View full text Add to dashboard Cite

Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper Provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this paper summarizes several interesting directions in the future research. We hope that this overview can shed light to MDPs in queues and networks, and also to their extensive applications in various practical areas.One main purpose of this paper is to provide an overview for research on MDPs in queues and networks in the last six decades. Also, such a survey is first related to several other basic studies, such as, Markov processes, queueing systems, queueing networks, Markov decision processes, sensitivity-based optimization, stochastic optimization, fluid and diffusion control. Therefore, our analysis begins from three simple introductions:1 Markov processes and Markov decision processes, queues and queueing networks, and queueing dynamic control. (a) Markov processes and Markov decision processesThe Markov processes, together with the Markov property, were first introduced by a Russian mathematician: Andrei Andreevich Markov in 1906. See Markov [238] for more details. From then on, as a basically mathematical tool, the Markov processes have extensively been discussed by many authors, e.g., see some excellent books by Doob [99], Karlin [175], Karlin and Taylor [176], Chung [80], Anderson [21], Kemeny et al. [181], Meyn and Tweedie [241], Chen [77], Ethier and Kurtz [110] and so on. In 1960, Howard [165] is the first to propose and discuss the MDP (or stochastic dynamic programming) in terms of his Ph.D thesis, which opened up a new and important field through an interesting intersection between Markov processes and dynamic programming (e.g., see Bellman and Kalaba [32]). From then on, not only are the MDPs an important branch in the area of Markov processes, but also it is a basic method in modern dynamic control theory. Crucially, the MDPs have been greatly motivated and widely applied in many practical areas in the past 60 years. Readers may refer to some excellent books, for example, the discrete-time MDPs by Puterman [261], Glasserman and Yao [143], Bertsekas [33], Bertsekas and Tsitsiklis [34], Hernádez-Lerma and Lasserre [155, 156], Altman [9], Koole [193] and Hu and Yue [166]; the continuous-time MDPs by Guo and Hernández-Lerma [145]; the partially observable MDPs by Cassandra [67] and Krishnamurthy [196]; the competitive MDPs (i.e., stochastic game) by [127]; the sensitivity-based optimization by Cao [58]; some applications of MDPs by Feinberg and Shwartz (Eds.) [122]and Boucherie and Van Dijk (Eds.) [44]; and so on. (b) Queues and queueing networksIn the early 20th century, a Danmark mathematician: Agner Krarup Erlang, published a pioneering work [109] of queueing theory in 1909, which started the study of queueing theory and traffic engineering. Over the past 100 years, queueing theory has been regarded as a key mathematical tool not only for analyzing practical s...

show abstract

Virtual allocation policies for many-server queues with abandonment

Long

Zhang

2019

Math Meth Oper Res

View full text Add to dashboard Cite

Scheduling a multi class queue with many exponential servers: asymptotic optimality in heavy traffic

Cited by 120 publications

References 34 publications

Optimal Control of Markov-Modulated Multiclass Many-Server Queues

Optimal Control of Markov-Modulated Multiclass Many-Server Queues

An Overview for Markov Decision Processes in Queues and Networks

Virtual allocation policies for many-server queues with abandonment

Contact Info

Product

Resources

About