Solving Simple Stochastic Games with Few Coin Toss Positions

Abstract

Gimbert and Horn gave an algorithm for solving simple stochastic games with running time O(r!n) where n is the number of positions of the simple stochastic game and r is the number of its coin toss positions. Chatterjee et al. pointed out that a variant of strategy iteration can be implemented to solve this problem in time 4 r r O(1) n O(1) . In this paper, we show that an algorithm combining value iteration with retrograde analysis achieves a time bound of O(r2 r (r log r + n)), thus improving both time bounds. While the algorithm is simple, the analysis leading to this time bound is involved, using techniques of extremal combinatorics to identify worst case instances for the algorithm.

A Pseudo-Polynomial Algorithm for Mean Payoff Stochastic Games with Perfect Information and a Few Random Positions

“…gave a randomized algorithm with expected running time |V R |!|V | O(1) . Ibsen-Jensen and Miltersen [IJM12] improved these bounds by showing that a variant of value iteration solves SSGs in time in O(|V R |2 |VR| (|V R | log |V R | + |V |)). For BW-games several pseudo-polynomial and subexponential algorithms are known [GKK88, KL93, ZP96, Pis99, BV01a, BV01b, HBV04, BV05, BV07, Hal07, Vor08]; see also [JPZ06] for parity games.…”

Strategy Complexity of Finite-Horizon Markov Decision Processes and Simple Stochastic Games

“…Proof Since there is an optimal Markov strategy, there is a counter-based strategy, which uses memory at most log T . As shown by Ibsen-Jensen and Miltersen [5] for any game G T , if the horizon is greater than 2 log ǫ −1 2 n , the value of G T approximates the value of G with in ǫ. It is clear that the value of all states are the same in an infinite-horizon game if either player is forced to play an optimal strategy.…”

Approximation Schemes for Stochastic Mean Payoff Games with Perfect Information and Few Random Positions

“…There are various improvements with smaller dependence on k [9,15,20,23] (note that even though BWR-games are polynomially reducible to simple stochastic games, under this reduction the number of random positions does not stay constant, but is only polynomially bounded in n, even if the original BWRgame had a constant number of random positions). Recently, a pseudo-polynomial algorithm was given for BWR-games with a constant number of random positions and polynomial common denominator of transition probabilities, but under the assumption that the game is ergodic (that is, the value does not depend on the ini-tial position) [5].…”