A risk minimization problem for finite horizon semi-Markov decision processes with loss rates

Abstract

This paper deals with the risk probability for finite horizon semi-Markov decision processes with loss rates. The criterion to be minimized is the risk probability that the total loss incurred during a finite horizon exceed a loss level. For such an optimality problem, we first establish the optimality equation, and prove that the optimal value function is a unique solution to the optimality equation. We then show the existence of an optimal policy, and develop a value iteration algorithm for computing the value function and optimal policies. We also derive the approximation of the value function and the rules of iteration. Finally, a numerical example is given to illustrate our results.

Risk probability optimization problem for finite horizon continuous time Markov decision processes with loss rate

“…Moreover, in some control models such as economic and financial systems, the controller is often focused on the probability that the total loss incurred over a given time horizon exceeds the initial capacity. Hence, limited literature [13,19] is available for the loss case, which minimizes the risk probability P π (B l > λ) over all the policies π, where B l denotes the total loss during a given time horizon, λ denotes the loss level (or goal). Specifically, Huang, Zou, and Guo [13] investigate the loss rates risk probability for first passage SMDPs, They use the invariant embedding technique to establish the optimality equation and prove the existence of optimal risk probability policies.…”

“…Specifically, Huang, Zou, and Guo [13] investigate the loss rates risk probability for first passage SMDPs, They use the invariant embedding technique to establish the optimality equation and prove the existence of optimal risk probability policies. Similarly, Liu and Zou [19] consider the risk probability criterion for finite horizon SMDPs with loss rate using the idea and iteration technique in [14] to demonstrate that the value function satisfies the optimality equation and the existence of optimal policies, and to derive an efficient algorithm for solving the value function. A review of the above mentioned literature demonstrates that the risk probability criterion with loss rate just considered in SMDPs, and CTMDPs for the risk probability criterion with the loss rate have not yet been explored.…”