Heavy Traffic Convergence of a Controlled, Multiclass Queueing System

Martins, Luiz Eduardo Galvão; Shreve, Steven E.; Soner, H. Meté

doi:10.1137/s0363012994265882

Cited by 75 publications

(53 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, for many singular control problems in reversible investment and in areas such as queuing and wireless communications (Martins, Shreve, and Soner (1996); Assaf (1997); Harrison and Van Mieghem (1997); Ata, Harrison, and Shepp (2005)), there is no regularity for either the value function or the boundaries. Therefore, two important mathematical issues remain: 1) necessary conditions for regularity properties; and 2) characterization for the value function and for the action and no-action regions when these regularity conditions fail.…”

Section: Introductionmentioning

confidence: 99%

A Class of Singular Control Problems and the Smooth Fit Principle

Guo¹,

Tomecek

2009

SIAM J. Control Optim.

View full text Add to dashboard Cite

This paper analyzes a class of singular control problems for which value functions are not necessarily smooth. Necessary and sufficient conditions for the well-known smooth fit principle, along with the regularity of the value functions, are given. Explicit solutions for the optimal policy and for the value functions are provided. In particular, when payoff functions satisfy the usual Inada conditions, the boundaries between action and no-action regions are smooth and strictly monotonic as postulated and exploited in the existing literature (Dixit and Pindyck (1994)

show abstract

Section: Introductionmentioning

confidence: 99%

A Class of Singular Control Problems and the Smooth Fit Principle

Guo¹,

Tomecek

2009

SIAM J. Control Optim.

View full text Add to dashboard Cite

show abstract

“…In the context of scheduling and queueing problems, the relevant asymptotic regime is the heavy traffic limit, under which the stochastic model can be approximated by either a diffusion process or an ODE system; the limit models are named Brownian network or fluid approximation, depending on whether the limiting differential equation is stochastic or deterministic. Readers interested in the Brownian network approach may consult [23,42,40,36,37,59] and references therein. For the fluid approximation of stochastic queueing networks, we refer to [12,11,40]; cf.…”

Section: Introductionmentioning

confidence: 99%

Optimal control of Markov jump processes: Asymptotic analysis, algorithms and applications to the modeling of chemical reaction systems

Zhang

Hartmann

Kleist

2018

Communications in Mathematical Sciences

View full text Add to dashboard Cite

Markov jump processes are widely used to model natural and engineered processes. In the context of biological or chemical applications one typically refers to the chemical master equation (CME), which models the evolution of the probability mass of any copy-number combination of the interacting particles. When many interacting particles ("species") are considered, the complexity of the CME quickly increases, making direct numerical simulations impossible. This is even more problematic when one aims at controlling the Markov jump processes defined by the CME.In this work, we study both open loop and feedback optimal control problems of the Markov jump processes in the case that the controls can only be switched at fixed control stages. Based on Kurtz's limit theorems, we prove the convergence of the respective control value functions of the underlying Markov decision problem as the copy numbers of the species go to infinity. In the case of the optimal control problem on a finite time-horizon, we propose a hybrid control policy algorithm to overcome the difficulties due to the curse of dimensionality when the copy number of the involved species is large. Two numerical examples demonstrate the suitability of both the analysis and the proposed algorithms.

show abstract

“…Similarly, by β ≥ 0 we mean that β i ≥ 0, i = 1, 2, where β = (β 1 , β 2 ).] In the queueing network setting this parameter regime corresponds to Case IIB of [20]. In the notation of that paper, 3 correspond to the service rates and c 1 , c 2 , c 3 to the holding costs of the queueing network model.…”

mentioning

confidence: 99%

“…In the notation of that paper, 3 correspond to the service rates and c 1 , c 2 , c 3 to the holding costs of the queueing network model. The parameter regime α ≥ 0, β ≥ 0 (Case IIC of [20]) can be treated in a symmetric manner. Finally, the case α ≥ 0, β ≥ 0 (Case IID of [20]) appears to be a significantly harder problem and is beyond the scope of the current study.…”

mentioning

confidence: 99%

Optimal stopping and free boundary characterizations for some Brownian control problems

Budhiraja¹,

Ross²

2008

Ann. Appl. Probab.

View full text Add to dashboard Cite

A singular stochastic control problem with state constraints in twodimensions is studied. We show that the value function is C 1 and its directional derivatives are the value functions of certain optimal stopping problems. Guided by the optimal stopping problem, we then introduce the associated no-action region and the free boundary and show that, under appropriate conditions, an optimally controlled process is a Brownian motion in the noaction region with reflection at the free boundary. This proves a conjecture of Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133-2171] on the form of an optimal control for this class of singular control problems. An important issue in our analysis is that the running cost is Lipschitz but not C 1 . This lack of smoothness is one of the key obstacles in establishing regularity of the free boundary and of the value function. We show that the free boundary is Lipschitz and that the value function is C 2 in the interior of the no-action region. We then use a verification argument applied to a suitable C 2 approximation of the value function to establish optimality of the conjectured control.

show abstract

Heavy Traffic Convergence of a Controlled, Multiclass Queueing System

Cited by 75 publications

References 28 publications

A Class of Singular Control Problems and the Smooth Fit Principle

A Class of Singular Control Problems and the Smooth Fit Principle

Optimal control of Markov jump processes: Asymptotic analysis, algorithms and applications to the modeling of chemical reaction systems

Optimal stopping and free boundary characterizations for some Brownian control problems

Contact Info

Product

Resources

About