Blackwell Optimality for Controlled Diffusion Processes

Abstract

In this paper we study m-discount optimality (m ≥ −1) and Blackwell optimality for a general class of controlled (Markov) diffusion processes. To this end, a key step is to express the expected discounted reward function as a Laurent series, and then search certain control policies that lexicographically maximize the mth coefficient of this series for m = −1, 0, 1, . . . . This approach naturally leads to m-discount optimality and it gives Blackwell optimality in the limit as m → ∞.

Bias and Overtaking Equilibria for Zero-Sum Stochastic Differential Games

“…This is an extension of the example presented in [30], which in turn was motivated by the manufacturing system in [31]. A variant of this system was also studied in [18,19].…”

“…To obtain (33), first note that, by (30) and (6), h π 1 ,π 2 is indeed μ π 1 ,π 2 -integrable for every (π 1 , π 2 ) ∈ Π 1 × Π 2 . Then in (33) choose the distribution of the initial state to be μ π 1 ,π 2 and so (33) follows from Fubini's theorem and the invariance of μ π 1 ,π 2 .…”

Strong n-discount and finite-horizon optimality for continuous-time Markov decision processes

“…In fact, the EAR criterion does not distinguish two different average optimal policies that have different finite-horizon rewards. To avoid this situation, many authors consider more sensitive criteria such as (i) the variance-minimization criterion (see, for instance [2,9,10] and their references); (ii) overtaking optimality criteria, such as Flynn's 'opportunity cost', Dutta's criterion, and bias optimality, etc (see [2,[11][12][13][14][15] for instance); and (iii) 'discount-sensitive' criteria (see [14,[16][17][18][19]). …”