Distributed Fictitious Play in Potential Games with Time Varying Communication Networks

Abstract

We propose a distributed algorithm for multiagent systems that aim to optimize a common objective when agents differ in their estimates of the objective-relevant state of the environment. Each agent keeps an estimate of the environment and a model of the behavior of other agents. The model of other agents' behavior assumes agents choose their actions randomly based on a stationary distribution determined by the empirical frequencies of past actions. At each step, each agent takes the action that maximizes its expectation of the common objective computed with respect to its estimate of the environment and its model of others. We propose a weighted averaging rule with non-doubly stochastic weights for agents to estimate the empirical frequency of past actions of all other agents by exchanging their estimates with their neighbors over a time-varying communication network. Under this averaging rule, we show agents' estimates converge to the actual empirical frequencies fast enough. This implies convergence of actions to a Nash equilibrium of the game with identical payoffs given by the expectation of the common objective with respect to an asymptotically agreed estimate of the state of the environment.

Decentralized Fictitious Play Converges Near a Nash Equilibrium in Near-Potential Games

“…Assumption 2(iii) assures that the weights W j,t , defined as the collective update weights on agent j's empirical frequency [W j,t ] i,l = w i jl,t , is row stochastic for all times. Now, we state the convergence of local estimates ν i j,t to empirical frequencies f j,t -see [12] for the proof.…”

“…Lemma 1 (Proposition 1, [12]) Suppose Assumptions 1-2 hold. If f j0 = υ i j,0 holds for all pairs of agents j ∈ N and i ∈ N , then the local copies {υ i t } i∈N t≥0 converge to the empirical frequencies {f t } t≥0 with rate O(log t/t), i.e., ||υ i j,t − f j,t || = O(log t/t) for all j ∈ N and i ∈ N .…”

Decentralized Fictitious Play with Voluntary Communication in Random Communication Networks

“…FP is not a decentralized algorithm, since agents need to observe past actions of everyone to be able to keep track of empirical frequencies, and compute the expectations of their utilities. Recent works [16]- [19] consider a decentralized form of the fictitious play, in which agents form estimates on empirical frequencies of other agents' actions by averaging the estimates received from their neighbors in a communication network. These algorithms are shown to converge to a NE in weakly acyclic games, i.e., games that admit finite best-response improvement paths.…”

“…A 1 (i) and a i ∈ A 2 (i) such that, u i (a i , a −i ) − µ < u i (a i , a −i ). (19) for some µ > 0 satisfying (14). Note that ( 16) holds for both actions a i ∈ A 1 (i) and a i ∈ A 2 (i),…”

“…Next, we add u i (a i , f i −i (t)) and u i (a i , f i −i (t)) to the left and right hand sides of (19), respectively. Similarly, we subtract the same corresponding terms from the left and right hand sides of (19). Using the bounds in (20) and (21), we get…”