Joint Strategy Fictitious Play With Inertia for Potential Games

Abstract

Abstract-We consider multi-player repeated games involving a large number of players with large strategy spaces and enmeshed utility structures. In these "large-scale" games, players are inherently faced with limitations in both their observational and computational capabilities. Accordingly, players in large-scale games need to make their decisions using algorithms that accommodate limitations in information gathering and processing. This disqualifies some of the well known decision making models such as "Fictitious Play" (FP), in which each player must monitor the individual actions of every other player and must optimize over a high dimensional probability space. We will show that Joint Strategy Fictitious Play (JSFP), a close variant of FP, alleviates both the informational and computational burden of FP. Furthermore, we introduce JSFP with inertia, i.e., a probabilistic reluctance to change strategies, and establish the convergence to a pure Nash equilibrium in all generalized ordinal potential games in both cases of averaged or exponentially discounted historical data. We illustrate JSFP with inertia on the specific class of congestion games, a subset of generalized ordinal potential games. In particular, we illustrate the main results on a distributed traffic routing problem and derive tolling procedures that can lead to optimized total traffic congestion. Index Terms-Fictitious play (FP), joint strategy fictitious play (JSFP).

Evolutionary Game Theory: A Renaissance

“…A learning model is fitted to the data that has two components. (i) Suppliers of FR form beliefs about the prices offered by other suppliers through a fictitious play dynamic that, similarly to Marden et al [226] (see Section 8.4), assumes correlation in the prices offered by the other suppliers. This dynamic is discounted so that recent periods are weighted more heavily.…”

“…In this model there can be a multiplicity of stochastically stable quantities in between the Nash equilibrium and Walrasian quantities. The reason for this multiplicity is that memory is assortative in the sense that strategies at period t generated the payoffs they generated due to other strategies at period t. If every player has played the Nash equilibrium quantity for as long as they can remember, and a player makes an error and plays a higher quantity, this will reduce the payoffs associated with the Nash equilibrium quantity in that period, but will not affect the payoffs associated with the Nash equilibrium quantity in previous periods (a similar assortativity across time is generated in a best response setting by a variant of fictitious play in Marden et al [226], discussed in Section 8.4). This makes quantities lower than the Walrasian quantity more robust in the presence of memory.…”

“…For example, Marden et al [226] give a version of fictitious play in which a player, rather than considering the historical distribution of play by each of his opponents separately, instead assesses the effectiveness of actions against entire action profiles (excluding his own action) played in past periods, effectively assuming correlation in opponents' play. In finite potential games, this dynamic is shown to converge almost surely to a pure Nash equilibrium.…”

Fast convergence for time-varying semi-anonymous potential games

“…(vi) There exists a constant Λ ≥ 8c 0 ε −2 e 3β (6βλ + e β k(s − 1)) 2 (10) such that, for any t, t with |t − t | ≤ Λ,…”

“…A rich body of literature has focused on characterizing emergent global behavior for various classes of learning rules, e.g., fictitious play [6], [10], [14], regret based learning [11], and log-linear learning [1], [3], [12], [19]. Log-linear learning is of particular importance because it guarantees convergence to a potential function optimizer for the class of potential games.…”