Characterizations of Overtaking Optimality for Controlled Diffusion Processes

Blackwell Optimality for Controlled Diffusion Processes

“…Hence, we have two trivial cases: if h 0 is strictly increasing (h 0 > 0) or decreasing (h 0 < 0), then the control policy f a (x) ≡ a or, respectively f 0 (x) ≡ 0 is the unique policy that attains the maximum in (6.2); in other words, F −1 is the singleton {f a } or {f 0 }. This set coincides with the set of average optimal policies, according to Theorem 3.3 of [18] (see also [2], [4], and [10]). Now suppose that h 0 attains a maximum, say x 0 .…”

“…In Section 2 we introduce the control system and our main assumptions. In addition, we define the optimality criteria we are concerned with, and we summarize some known results on the Hamilton-Jacobi-Bellman (HJB) equation [2], [4], [8], [10], [17], [18], which is essentially our point of departure to analyze m-discount optimality and Blackwell optimality. In Section 3 we express the expected α-discounted v-reward (see (3.6)) for some function v as a Laurent series (see (3.11)).…”

“…Also, note that the functionv(f ) is the gainr(f ) in (2.12). In general, it can be seen by induction that h 18) and so, h k f is the bias when the reward rate is −h…”

Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

“…For continuous-time models, however, just a few references deal with this issue. For instance, Puterman [18] studied controlled diffusions on compact intervals and Jasso-Fuentes and Hernández-Lerma [12] considered general controlled diffusions. Regarding jump processes with nonfinite state space, Prieto-Rumeau and Hernández-Lerma [16] analyzed the case of a denumerable state space.…”

“…[11], [17], and [23]), the bias and the overtaking optimality criteria (that choose an average optimal policy with the maximal expected reward growth as the time horizon goes to ∞; see, e.g. [7], [8], [10, p. 132], [12], [16], and [19,Chapter 10]), and the so-called discountsensitive criteria (which choose policies that are asymptotically optimal as the discount rate converges to 0; see [7], [13], [15], [19,Chapter 10], and [22]), among others.…”