Distributed gradient methods to reach a Nash equilibrium in potential games

Abstract

We study multi-agent optimization problems described by ordinal potential games. The objective is to reach a Nash equilibrium (NE) in a distributed manner. It is well known that an asynchronous best response dynamics (ABRD) will always converge to a pure NE in this case. However, computing the exact best response at every step of the algorithm may be computationally heavy, if not impossible. Therefore, instead of computing the exact best response, we propose an algorithm that performs a "better response", which decreases the local cost rather than minimizing it. The agents perform a distributed asynchronous gradient descent algorithm, in which only a finite number of iterations of the gradient descent are performed by each player. We prove that this algorithm always converges to a NE and demonstrate via simulations that the computational time to reach the NE can be much shorter than with the classical ABRD. Taking into account the time required for each agent to communicate, the proposed algorithm is shown to also outperform a distributed synchronous gradient descent in simulations. I. INTRODUCTIONGame theory has been widely applied in various fields like economics [1], wireless communication [2] and automatic control [3]. A special class of games called potential games was introduced and studied by Monderer in [4], and several conditions guaranteeing the existence of a pure Nash equilibrium (NE) were provided. Potential games find many applications, e.g., in traffic management, wireless design, model predictive control, see [5].In [6], it was shown that in potential games, the asynchronous best response dynamics (ABRD), always converges to a pure NE. Recall that the ABRD involves players updating their actions to be one that minimizes their local cost asynchronously, while the other player actions are fixed. However, in many situations, the exact best responses which are obtained by minimizing a certain cost are hard or impossible to compute, and instead, algorithms may be used to find approximate best responses. Recent works have studied the convergence of approximate best response dynamics for special applications. In [7], a framework for iterative approximate best response algorithms is provided for distributed constraint optimization problems. In [8], the authors focus on approximate best response dynamics in interference games, which are a special class of games often studied in wireless communication. On the other hand, [9] looks at stochastic algorithms which are used to approximate the NE.