Significance Nash equilibrium, of central importance in strategic game theory, exists in all finite games. Here we prove that it exists also in all infinitely repeated games, with a finite or countably infinite set of players, in which the payoff function is bounded and measurable and the payoff depends only on what is played in the long run, i.e., not on what is played in any fixed finite number of stages. To this end we combine techniques from stochastic games with techniques from alternating-move games with Borel-measurable payoffs.

We consider discrete-time Markov decision processes in which the decision maker is interested in long but finite horizons. First we consider reachability objective: the decision maker’s goal is to reach a specific target state with the highest possible probability. A strategy is said to overtake another strategy, if it gives a strictly higher probability of reaching the target state on all sufficiently large but finite horizons. We prove that there exists a pure stationary strategy that is not overtaken by any pure strategy nor by any stationary strategy, under some condition on the transition structure and respectively under genericity. A strategy that is not overtaken by any other strategy, called an overtaking optimal strategy, does not always exist. We provide sufficient conditions for its existence. Next we consider safety objective: the decision maker’s goal is to avoid a specific state with the highest possible probability. We argue that the results proven for reachability objective extend to this model.

No abstract

We study the optimal use of information in Markov games with incomplete information on one side and two states. We provide a finite-stage algorithm for calculating the limit value as the gap between stages goes to 0, and an optimal strategy for the informed player in the limiting game in continuous time. This limiting strategy induces an ǫ-optimal strategy for the informed player, provided the gap between stages is small. Our results demonstrate when the informed player should use his information and how.

A real-valued function ϕ that is defined over all Borel sets of a topological space is regular if for every Borel set W, ϕ(W) is the supremum of ϕ(C), over all closed sets C that are contained in W, and the infimum of ϕ (O), over all open sets O that contain W.We study Blackwell games with finitely many players. We show that when each player has a countable set of actions and the objective of a certain player is represented by a Borel winning set, that player's minmax value is regular.We then use the regularity of the minmax value to establish the existence of ε-equilibria in two distinct classes of Blackwell games. One is the class of n-player Blackwell games where each player has a finite action space and an analytic winning set, and the sum of the minmax values over the players exceeds n − 1. The other class is that of Blackwell games with bounded upper semi-analytic payoff functions, history-independent finite action spaces, and history-independent minmax values. For the latter class, we obtain a characterization of the set of equilibrium payoffs.