Constrained Continuous-Time Markov Decision Processes on the Finite Horizon

Realizable Strategies in Continuous-Time Markov Decision Processes

“….} [10,11,12,13,18,20,24] which is called below as a π-strategy. On the other hand, if the relaxations are absent, that is, all kernels π n are degenerate and concentrated at singletons…”

“…In many works, the admissible sets of actions A(x) in the states x ∈ X were introduced [10,11,12,13,18,20,24,26,27,28]. In this connection, one can, e.g., put c 0 n (x, a) = +∞ in case action a is not admissible in the state x.…”

“…If Ξ = {ξ} is a singleton, then the argumentξ can be omitted everywhere, and the model transforms to the classical continuous-time Markov decision process [10,11,12,13,18,20,24]. In this case, by Corollary 1(b), a strategy is realizable if and only if it is indistinguishable from a randomized strategy which is defined by functions φ n (h n−1 , u) and is actually deterministic.…”

“…To distinguish from the case 2, we call such π-strategies "relaxed" rather than randomized. They became very popular in the last decade: see, e.g., [10,11,12,13,20,24] and the references therein. In Section 3 we explain what it means that the strategy defined in terms of π can be equivalently represented as a random process A(•).…”