Constrained Markov decision processes with first passage criteria

First passage Markov decision processes with constraints and varying discount factors

“…Note that, Assumption 1 (c)-(e) are the continuity-compactness conditions commonly assumed to prove the solvability of optimization problems as in [12,[14][15][16]23,24].…”

“…As is well known, as a generalization of the standard (infinite horizon) discounted DTMDPs as in [14,15], the first passage model of discounted DTMDPs has been widely studied in [5,8,16,24] for proving the existence of an optimal policy. In general, the main goal of this kind of study is to maximize (or minimize) the expected total discounted reward obtained (or cost paid) during a first passage time to some target set.…”

“…These convergence results are crucial for the finite approximation problems of optimal value and policies. Concerning the constrained DTMDPs with varying discount factors, for the single-constraint case, [16] showed the existence of a constrained optimal policy for the first passage criteria of DTMDPs with state-action dependent discount factors and denumerable states. For the multi-constraints case, [25] developed the convex analytic approach to the constrained DTMDPs in Borel state and action spaces with varying discount factors and bounded costs from below, and proved the existence of an optimal policy under slightly stronger conditions.…”

“…Inspired by the ideas in [3,10,12,16,24], we also introduce the concept of an occupation measure of a policy for the model of the first passage DTMDPs. Then, we characterize the occupation measure of a policy by a form of an equation, and obtain some properties of the occupation measures.…”

First passage risk probability optimality for continuous time Markov decision processes

“…Inspired by these situations, the first passage performance criteria are introduced into the MDPs. The literature on the first passage optimality problems for MDPs can be classified into two groups: (i) One focuses on the first passage expected criterion (see, for instance [5,11,19,26,12,6,15]), which means that the expected total rewards during a rand time that the state process first enters a given target set. This criterion can usually be regarded as a generalization of the standard criterion [3,4,8,9,22].…”