Risk-sensitive semi-Markov decision processes with general utilities and multiple criteria

Abstract

In this paper we investigate risk-sensitive semi-Markov decision processes with a Borel state space, unbounded cost rates, and general utility functions. The performance criteria are several expected utilities of the total cost in a finite horizon. Our analysis is based on a type of finite-horizon occupation measure. We express the distribution of the finite-horizon cost in terms of the occupation measure for each policy, wherein the discount is not needed. For unconstrained and constrained problems, we establish the existence and computation of optimal policies. In particular, we develop a linear program and its dual program for the constrained problem and, moreover, establish the strong duality between the two programs. Finally, we provide two special cases of our results, one of which concerns the discrete-time model, and the other the chance-constrained problem.

Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space

“…Proof (i) Let (x, a) ∈ K be arbitrary but fixed, and note that (13) yields that e λV αn ,εn (x) ≤ e λC(…”

“…On the other hand, the literature on semi-Markov models with risksensitive criteria is somewhat scarce. In [5] (uncontrolled) semi-Markov chains were studied and the risk-sensitive average cost was characterized via a Poisson equation, a result that was generalized to the controlled case in [6], whereas semi-Markov models with a finite decision horizon are studied in [13].…”

Risk-sensitive average continuous-time Markov decision processes with unbounded transition and cost rates

“…In comparison, there has been limited work on risk-sensitive continuous-time Markov decision processes (CTMDPs). For background on risk-sensitive CTMDPs, see [7,11,22] for the risk-sensitive (infinite-horizon) discounted criterion, [7,22,23,25,26,27] for the risk-sensitive (long-run) average criterion, [7,9,14,17,18,23,24] for the risk-sensitive finite-horizon criterion, and [8,28] for the risk-sensitive total cost criterion. In this paper we will further consider the risksensitive average criterion for CTMDPs studied in [7,22,23,25,26,27].…”

Risk-sensitive Semi-Markov Decision Problems with Discounted Cost and General Utilities

“…There the state space is assumed to be finite and the sojourn time distributions are assumed to have a compact support. In [16], the authors consider risk-sensitive control of semi-Markov processes on a fixed finite horizon [0, T ]. To the best of our knowledge, this is first paper dealing with risk-sensitive control of semi-Markov process for a discounted cost criterion over the infinite horizon.…”