On Completely Mixed Stochastic Games

Abstract

In this paper, we consider a zero-sum undiscounted stochastic game which has finite state space and finitely many pure actions. Also, we assume the transition probability of the undiscounted stochastic game is controlled by one player and all the optimal strategies of the game are strictly positive. Under all the above assumptions, we show that the β-discounted stochastic games with same payoff matrices and β sufficiently close to 1 are also completely mixed. We also provide a necessary condition under which the individual matrix games are completely mixed. We also show that, if we have non-zero value in some state for the undiscounted stochastic game then for β sufficiently close to 1 the β-discounted stochastic game also possess nonzero value in the same state.

Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games

“…If the original Markov game has at least one completely mixed Nash equilibrium, the Nash equilibrium of the regularized Markov game should also be completely mixed even when the regularization weight is small, since the regularization encourages the solution to be more uniform. The reward function that results in completely mixed Markov games is well studied in Raghavan [1978], Kaplansky [1995], Das et al [2017].…”