Ergodic Control, Bias, and Sensitive Discount Optimality for Markov Diffusion Processes

Jasso-Fuentes, Héctor; Hernández–Lerma, Onésimo

doi:10.1080/07362990802679034

Cited by 13 publications

(20 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [23], the authors let T → ∞ in (20) and (21) to derive that f * is gain optimal. Then, since F ao = F ca when the state space is denumerable, the stated result follows.…”

Section: Definition 33mentioning

confidence: 99%

“…For the case of continuous-time MDPs, however, just a few references deal with this issue. For instance, [20] studies the results 2)-4) for controlled diffusions in compact intervals and [21] considers general controlled diffusions. Regarding jump processes with nonfinite state space, [22] and [23] analyze results 2)-4) and 1) for the case of a denumerable state space, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Strong n-discount and finite-horizon optimality for continuous-time Markov decision processes

Zhu

Guo

2014

J Syst Sci Complex

View full text Add to dashboard Cite

This paper studies the strong n(n = −1, 0)-discount and finite horizon criteria for continuoustime Markov decision processes in Polish spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under mild conditions, the authors prove the existence of strong n(n = −1, 0)-discount optimal stationary policies by developing two equivalence relations: One is between the standard expected average reward and strong −1-discount optimality, and the other is between the bias and strong 0-discount optimality. The authors also prove the existence of an optimal policy for a finite horizon control problem by developing an interesting characterization of a canonical triplet.

show abstract

“…In [23], the authors let T → ∞ in (20) and (21) to derive that f * is gain optimal. Then, since F ao = F ca when the state space is denumerable, the stated result follows.…”

Section: Definition 33mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Strong n-discount and finite-horizon optimality for continuous-time Markov decision processes

Zhu

Guo

2014

J Syst Sci Complex

View full text Add to dashboard Cite

show abstract

“…On the other hand, for continuous-time models, Puterman [27] considered one-dimensional diffusion processes with values in a compact interval, whereas Prieto-Rumeau and Hernández-Lerma [25] and Prieto-Rumeau [23] considered continuous-time controlled Markov chains with a denumerable state space. Finally, Jasso-Fuentes and Hernández-Lerma [19] studied special cases of strong m-discount optimality for the cases m = −1, 0 for controlled diffusion processes. Our approach in this paper is mainly based on [19] and [23].…”

Section: Dx(t) = B(x(t) U(t)) Dt + σ (X(t)) Db(t) For All T ≥ 0 and mentioning

confidence: 99%

“…Finally, Jasso-Fuentes and Hernández-Lerma [19] studied special cases of strong m-discount optimality for the cases m = −1, 0 for controlled diffusion processes. Our approach in this paper is mainly based on [19] and [23].…”

Section: Dx(t) = B(x(t) U(t)) Dt + σ (X(t)) Db(t) For All T ≥ 0 and mentioning

confidence: 99%

Blackwell Optimality for Controlled Diffusion Processes

Jasso-Fuentes

Hernández–Lerma

2009

J. Appl. Probab.

Self Cite

View full text Add to dashboard Cite

In this paper we study m-discount optimality (m ≥ −1) and Blackwell optimality for a general class of controlled (Markov) diffusion processes. To this end, a key step is to express the expected discounted reward function as a Laurent series, and then search certain control policies that lexicographically maximize the mth coefficient of this series for m = −1, 0, 1, . . . . This approach naturally leads to m-discount optimality and it gives Blackwell optimality in the limit as m → ∞.

show abstract

“…[11], [17], and [23]), the bias and the overtaking optimality criteria (that choose an average optimal policy with the maximal expected reward growth as the time horizon goes to ∞; see, e.g. [7], [8], [10, p. 132], [12], [16], and [19,Chapter 10]), and the so-called discountsensitive criteria (which choose policies that are asymptotically optimal as the discount rate converges to 0; see [7], [13], [15], [19,Chapter 10], and [22]), among others.…”

Section: Introductionmentioning

confidence: 99%

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

Zhu

Prieto-Rumeau

2008

J. Appl. Probab.

View full text Add to dashboard Cite

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

show abstract

Ergodic Control, Bias, and Sensitive Discount Optimality for Markov Diffusion Processes

Cited by 13 publications

References 22 publications

Strong n-discount and finite-horizon optimality for continuous-time Markov decision processes

Strong n-discount and finite-horizon optimality for continuous-time Markov decision processes

Blackwell Optimality for Controlled Diffusion Processes

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

Contact Info

Product

Resources

About