Partially observable queueing systems with controlled service rates under a discounted optimality criterion

This paper deals with a class of partially observable discounted Markov decision processes defined on Borel state and action spaces, under unbounded one-stage cost. The discount rate is a stochastic process evolving according to a difference equation, which is also assumed to be partially observable. Introducing a suitable control model and filtering processes, we prove the existence of optimal control policies. In addition, we illustrate our results in a class of GI/GI/1 queueing systems where we obtain explicitly the corresponding optimality equation and the filtering process.

show abstract

“…We will borrow the recursive equation for g 1 t from [10], by changing what has to be changing as follows.…”

Section: Recursive Evolution Of Densitiesmentioning

confidence: 99%

“…P r o o f . The continuity of the function (80) is proved following similar arguments as in[10, Lemma 6.2]. Now, let {ϕ n } be a sequence in D d such that ϕ n → ϕ ∈ D d (see (48)).…”

mentioning

confidence: 96%

Partially observable Markov decision processes with partially observable random discount factors

Martinez-Garcia¹,

Minjárez‐Sosa²,

Vega-Amaya³

2023

Kybernetika

View full text Add to dashboard Cite

show abstract

“…The notation I{A} stands for the indicator of random event A and F (t) def = 1 − F (t). Comment shortly the formulae ( 7)- (8). Indeed, if the first message leaves the system at time y < t and the number of messages which arrive until y equals 0 ≤ i ≤ K − n − 1, then the system "renews" the operation at time y with exactly n+i−1 messages present and must serve m − 1 messages during time period of length t − y (the first summand on the right side of ( 7)-( 8)).…”

Section: Departure Process During Busy Periodmentioning

confidence: 99%

Non-stationary departure process in a batch-arrival queue with finite buffer capacity and threshold-type control mechanism

Kempa¹,

Kurzyk²

2022

Kybernetika

View full text Add to dashboard Cite

Non-stationary behavior of departure process in a finite-buffer M X /G/1/K-type queueing model with batch arrivals, in which a threshold-type waking up N -policy is implemented, is studied. According to this policy, after each idle time a new busy period is being started with the N th message occurrence, where the threshold value N is fixed. Using the analytical approach based on the idea of an embedded Markov chain, integral equations, continuous total probability law, renewal theory and linear algebra, a compact-form representation for the mixed double transform (probability generating function of the Laplace transform) of the probability distribution of the number of messages completely served up to fixed time t is obtained. The considered queueing system has potential applications in modeling nodes of wireless sensor networks (WSNs) with battery saving mechanism based on threshold-type waking up of the radio. An illustrating simulational and numerical study is attached.

show abstract

Partially observable queueing systems with controlled service rates under a discounted optimality criterion

Abstract: We are concerned with a class of GI/GI/1 queueing systems with controlled service rates, in which the waiting times are only observed when they take zero value. Applying a suitable filtering process, we show the existence of optimal control policies under a discounted optimality criterion.

Cited by 2 publications

References 12 publications

Partially observable Markov decision processes with partially observable random discount factors

Partially observable Markov decision processes with partially observable random discount factors

Non-stationary departure process in a batch-arrival queue with finite buffer capacity and threshold-type control mechanism

Contact Info

Product

Resources

About