The Complexity of Solving Stochastic Games on Graphs

Abstract

Abstract. We consider some well-known families of two-player zero-sum perfect-information stochastic games played on finite directed graphs. Generalizing and unifying results of Liggett and Lippman, Zwick and Paterson, and Chatterjee and Henzinger, we show that the following tasks are polynomial-time (Turing) equivalent.-Solving stochastic parity games, -Solving simple stochastic games, -Solving stochastic terminal-payoff games with payoffs and probabilities given in unary, -Solving stochastic terminal-payoff games with payoffs and probabilities given in binary, -Solving stochastic mean-payoff games with rewards and probabilities given in unary, -Solving stochastic mean-payoff games with rewards and probabilities given in binary, -Solving stochastic discounted-payoff games with discount factor, rewards and probabilities given in binary. It is unknown whether these tasks can be performed in polynomial time. In the above list, "solving" may mean either quantitatively solving a game (computing the values of its positions) or strategically solving a game (computing an optimal strategy for each player). In particular, these two tasks are polynomial-time equivalent for all the games listed above. We also consider a more refined notion of equivalence between quantitatively and strategically solving a game. We exhibit a linear time algorithm that given a simple stochastic game or a terminal-payoff game and the values of all positions of that game, computes a pair of optimal strategies. Consequently, for any restriction one may put on the simple stochastic game model, quantitatively solving is polynomial-time equivalent to strategically solving the resulting class of games.

Tropically Convex Constraint Satisfaction

“…Such relations have been called tropically convex in the literature [12]. 2 This class is a non trivial extension of the max-atoms problem (for instance it contains relations such as x (y + z)/2), and it is not covered by the known reduction to mean payoff games [28,1,3]. Indeed, it is open whether the CSP for tropically convex semilinear relations can be reduced to mean payoff games (in fact, Zwick and Paterson [32] believe that mean payoff games are "strictly easier" than simple stochastic games, which reduce to our problem via the results presented in Section 4).…”

“…We show that a semilinear relation is max-closed if and only if it can be defined by a semilinear Horn formula, which we define as a finite conjunction of semilinear Horn clauses, this is, finite disjunctions of the form m i=1ā ix i c i 1 Interpretations in the sense of model theory; we refer to Hodges [20] since we do not need this concept further. 2 The original definition of tropical convexity is for the dual situation, considering min instead of max.…”

“…We apply (2) to the field of formal Laurent series F(( )) with coefficients in F. Each semilinear set X ⊂ F n has a unique extension X * to F(( )) n , which is the set defined by the same formula that defines X (clearly which one is chosen is immaterial). Let X denote our max-closed set, which is not necessarily closed.…”

“…By (2), it suffices to show that basic closed max-closed sets are primitive positive definable using our relations. Consider the set…”

“…Even though this field is active since the 70s, to the best of our knowledge, no application of its results to semilinear feasibility problems has been published yet. Note that to solve mean payoff stochastic games is in NP ∩ co-NP, as a consequence of the existence of optimal positional strategies; see [17] and [27] (for a general reference on the computational complexity of stochastic games, see [2]). However, we cannot use this fact directly, because stochastic games only relate to a subset of tropically convex sets.…”