The structure of non-zero-sum stochastic games

The challenge of non-zero-sum stochastic games

“…Simon (2007): if x is not in W = R n \{r | ∀j r j > 1} then the only equilibrium of Γ(x) is the one where all players choose to continue with certainty. The proof involves looking at any pathway P of vectors in R n such that one end of P lies slightly inside of W , the other end lies far outside of W , and the intersection of P with the boundary of W is only one point.…”

“…If there is only one state where the decision maker can have any influence, the conjecture is easy to prove (R.S. Simon 2007), and this is the context of quitting games.…”

“…The result was generalised further (R.S. Simon 2007), with the role of M and n replaced by the expectation of the total variation of changes in the future expected payoff. This result allows one to punish the decision maker for gaining cumulative advantage without punishing the honest decision maker unduly.…”

“…This question appears to be critical to understanding stochastic games, and it is posed in R.S. Simon (2007) as a conjecture. If there is only one state where the decision maker can have any influence, the conjecture is easy to prove (R.S.…”

“…Simon (2007): if a decision maker is asked to choose actions in a stationary way such that at each stage the induced future expected differences in payoff from these actions is never more than δ, then the probability that this player will gain cumulatively more than ǫ in payoff is no more than ǫ for the relationship δ = ǫ 3 nM , where n is the number of different states and M is the maximal difference in payoffs. The proof follows an application of the Doob Martingale inequality.…”