Nonzero-Sum Risk-Sensitive Stochastic Games on a Countable State Space

Abstract

The infinite horizon risk-sensitive discounted-cost and ergodic-cost nonzero-sum stochastic games for controlled Markov chains with countably many states are analyzed. For the discounted-cost game, we prove the existence of Nash equilibrium strategies in the class of Markov strategies under fairly general conditions. Under an additional geometric ergodicity condition and a small cost criterion, the existence of Nash equilibrium strategies in the class of stationary Markov strategies is proved for the ergodic-cost game.

Incomplete information and risk sensitive analysis of sequential games without a predetermined order of turns

“…To do this, the Arrow [2] and Pratt [14] approach for risk sensitivity is used and the case in which players only consider what's happening in the game is taken into account, without any external variables interfering with their decision making. Not only is this the usual approach taken when studying Markov decision processes, for instance, in [9] and [16], but it has also been applied in [3,4,8,10,12], and [13] to study risk sensitivity in static and dynamic games including, in this last case, Markov and differential games. For the games presented in this work, in order to ensure the existence of Nash equilibria the authors proceed by showing that an adequately defined set of best responses satisfies the conditions of Kakutani's fixed point theorem [6] and [11] (for an extension of the Kakutani's fixed point theorem known as Kakutani-Fan-Glicksberg's fixed point theorem, see [1]) and by observing that the fixed points are exactly the Nash equilibria for the games.…”

“…x 1 (B) =5 8 , and for player 2, any mixed strategies such that 0.4x 2 (C; α)+ 0.6x 2 (C; β) =3 4 . Now if player 1 is risk sensitive, his expected utility is given by:[e −6λ +e −6λ −e −5λ −e −4λ +(e −3λ +e −4λ −e −5λ −e −6λ )(0.4x 2 (C; α)+0.6x 2 (C; β))]x 1 (A) which changes the second equilibrium described above to x 1 (A) =3 8 , x 1 (B) = 5 8 and for player 2 any mixed strategies such that: 0.4x 2 (C; α) + 0.6x 2 (C; β) = e −4λ + e −5λ − 2e −6λ e −3λ + e −4λ − e −5λ − e −6λ .…”

Zero and Non-zero Sum Risk-sensitive Semi-Markov Games

“…Since the pioneering work of Howard and Matheson [14], there has been a lot of work on risk-sensitive control of both discrete and continuous time stochastic processes. Risk sensitive games for discrete time Markov chains has been studied by several authors, see for instance [3,4,7] for zerosum games and [2,18] for non-zero sum games. Risk-sensitive games for continuous-time diffusions has been studied in [6,11,12].…”

Model and Reinforcement Learning for Markov Games with Risk Preferences

“…Risk-sensitive/Robust Games Risk-sensitive games have already been considered in (Klompstra 2000;Ghosh, Kumar, and Pal 2016;Basu and Ghosh 2017;Bäuerle and Rieder 2017;Jose and Zhuang 2018). Risk-sensitivity refers to the specific certainty equivalent (1/θ) ln (E [exp (θ X)]) where θ > 0 is the risk sensitivity parameter.…”

“…Risk-sensitivity refers to the specific certainty equivalent (1/θ) ln (E [exp (θ X)]) where θ > 0 is the risk sensitivity parameter. (Ghosh, Kumar, and Pal 2016;Basu and Ghosh 2017) focus on zero-sum risksensitive games under continuous time setting.…”