Solutions of the average cost optimality equation for finite Markov decision chains: risk-sensitive and risk-neutral criteria

Risk-sensitive average optimality in Markov decision processes

“…In contrast to the risk-neutral model, if the resulting transition probability matrix P (f ) is unichain and contains also transient states, solution of equations (26)-(28) can be guaranteed only for the small values of the risk sensitivity coefficient (see e. g. [7,8,9,25]). Conditions guaranteeing existence of solutions to (26)- (27) were studied in many papers, see e. g. [3][4][5][6][7][8][9][10][11][12][25][26][27][28].…”

“…Observe that the spectral radius of ρ (N) (f ) of Q (NN) (f ) = 0 equals null if and only all diagonal elements of Q (NN) (f ) equal null. Then each transient state of P (f ) = [p ij (f i )] (i. e. each state of Q (NN) (f )) is absorbed in the recurrent class of P (f ) after a finite number of transitions (at most equal to the number of transient states), see[6,10].…”

Vanishing discount approximations in controlled Markov chains with risk-sensitive average criterion

“…For the risk-neutral case λ = 0, this is a classical result and its proof can be found in [17] or [22]. For the λ = 0 case, a verification of the above lemma can be found, for example, in [7]. Throughout the remainder of this paper, the state z ∈ S is fixed, and the pair ( Definition 2.3.…”

Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space

“…In addition to play an important role to establish the uniqueness result in Lemma 2.1(iii), Assumption 2.2 is also crucial to ensure the existence of a solution of the optimality equation (7); see [4,5] or [6].…”