Selfish Robustness and Equilibria in Multi-Player Bandits

Abstract

Motivated by cognitive radios, stochastic multi-player multi-armed bandits gained a lot of interest recently. In this class of problems, several players simultaneously pull arms and encounter a collision -with 0 reward -if some of them pull the same arm at the same time. While the cooperative case where players maximize the collective reward (obediently following some fixed protocol) has been mostly considered, robustness to malicious players is a crucial and challenging concern. Existing approaches consider only the case of adversarial jammers whose objective is to blindly minimize the collective reward.We shall consider instead the more natural class of selfish players whose incentives are to maximize their individual rewards, potentially at the expense of the social welfare. We provide the first algorithm robust to selfish players (a.k.a. Nash equilibrium) with a logarithmic regret, when the arm reward is observed. When collisions are also observed, Grim Trigger type of strategies enable some implicit communication-based algorithms and we construct robust algorithms in two different settings: in the homogeneous case (with a regret comparable to the centralized optimal one) and in the heterogeneous case (for an adapted and relevant notion of regret). We also provide impossibility results when only the reward is observed or when arm means vary arbitrarily among players.

Dominate or Delete: Decentralized Competing Bandits in Serial Dictatorship

“…In this section, we discuss robustness properties when any agent deviates from executing the UCB-D3 algorithm in order to maximize the collected reward. We consider a desirable robustness property called ε Nash Equilibrium for multi-agent algorithms, recently proposed in [8]. Roughly, this property guarantees that no agent can significantly increase (by at-most additive ε) its rewards by unilaterally deviating from the UCB-D3; protocol.…”

“…Roughly, this property guarantees that no agent can significantly increase (by at-most additive ε) its rewards by unilaterally deviating from the UCB-D3; protocol. Although this is a weaker concept compared to the classical Nash-Equilibrium used in the theory of repeated games [27], is nevertheless a useful property for practical algorithms in multi-agent bandits to posses [8].…”

“…The proof generalizes the framework in Chapter 15 of [23] for the multi-agent framework. 8 Canonical multi-agent bandit model: We now define the (N -agent, K-arm, T-horizon) bandit models. The canonical bandit model (N -agent, K-arm, T-horizon) lies in a measurable space {Ω, F}.…”