Optimal policies for constrained average-cost Markov decision processes

Abstract

We give mild conditions for the existence of optimal solutions for a Markov decision problem with average cost, under m constraints of the same kind, in Borel actions and states spaces. Moreover, there is an optimal policy that is a convex combination of at most m + 1 deterministic policies.

Joint Routing, Scheduling and Power Control Providing Hard Deadline in Wireless Multihop Networks

“…[32] implemented an online algorithm by modifying the value iteration equation that minimizes the average power under an average delay constraint for a single user. Existence of stationary optimal polices for the constrained average-cost Markov decision processes was shown by setting up the problem as a constrained Markov decision process (C-MDP) in ( [2], [13], [38]). Using techniques in Markov decision process (MDP), structural properties of the optimal policy were obtained in [12].…”

Joint routing, scheduling and power control providing hard deadline in wireless multihop networks

“…[32] implemented an online algorithm by modifying the value iteration equation that minimizes the average power under an average delay constraint for a single user. Existence of stationary optimal polices for the constrained average-cost Markov decision processes was shown by setting up the problem as a constrained Markov decision process (C-MDP) in ( [2], [13], [38]). Using techniques in Markov decision process (MDP), structural properties of the optimal policy were obtained in [12].…”

Finite horizon semi-Markov decision processes with multiple constraints

“…A large amount of literature has been developed for solving constrained MDPs; see, for instance, [1], [2], [5], [9], [14] on the discrete-time MDPs (DTMDPs), [7], [10], [11], [19] on the continuous-time jump MDPs (CTJMDPs), and [4], [6], [7] on the semi-Markov decision processes (SMDPs). In these references, most of them deal with constrained problems with infinite horizons, while only a few are related to those with finite horizons [5], [7], [17], [18].…”