Robust Multi-Agent Bandits Over Undirected Graphs

Abstract

We consider a multi-agent multi-armed bandit setting in which n honest agents collaborate over a network to minimize regret but m malicious agents can disrupt learning arbitrarily. Assuming the network is the complete graph, existing algorithms incur O((m + K/n) log(T )/∆) regret in this setting, where K is the number of arms and ∆ is the arm gap. For m K, this improves over the single-agent baseline regret of O(K log(T )/∆).In this work, we show the situation is murkier beyond the case of a complete graph. In particular, we prove that if the state-of-the-art algorithm is used on the undirected line graph, honest agents can suffer (nearly) linear regret until time is doubly exponential in K and n. In light of this negative result, we propose a new algorithm for which the i-th agent has regret O((d mal (i) + K/n) log(T )/∆) on any connected and undirected graph, where d mal (i) is the number of i's neighbors who are malicious. Thus, we generalize existing regret bounds beyond the complete graph (where d mal (i) = m), and show the effect of malicious agents is entirely local (in the sense that only the d mal (i) malicious agents directly connected to i affect its long-term regret).

Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Bounds

“…The main focus in these papers is the design of coordination protocols among the agents that balance communication-efficiency with performance. A few very recent works [56][57][58][59] also look at the effect of attacks, but for the simpler unstructured multi-armed bandit problem [60]. Accounting for adversarial agents in the structured linear bandit setting we consider here requires significantly different ideas that we develop in this paper.…”