Cooperative Online Learning: Keeping your Neighbors Updated

Abstract

We study an asynchronous online learning setting with a network of agents. At each time step, some of the agents are activated, requested to make a prediction, and pay the corresponding loss. The loss function is then revealed to these agents and also to their neighbors in the network. Our results characterize how much knowing the network structure affects the regret as a function of the model of agent activations. When activations are stochastic, the optimal regret (up to constant factors) is shown to be of order √ αT , where T is the horizon and α is the independence number of the network. We prove that the upper bound is achieved even when agents have no information about the network structure. When activations are adversarial the situation changes dramatically: if agents ignore the network structure, a Ω(T ) lower bound on the regret can be proven, showing that learning is impossible. However, when agents can choose to ignore some of their neighbors based on the knowledge of the network structure, we prove a O( √ χT ) sublinear regret bound, where χ ≥ α is the clique-covering number of the network.

Kernel Methods for Cooperative Multi-Agent Contextual Bandits

“…Both these regret bounds match the rates mentioned for the context-free case in (Cesa-Bianchi et al, 2019b;Bar-On & Mansour, 2019). Moreover, when γ = 1, then the bound on the group regret matches the lower bound shown in the nonstochastic case (Cesa-Bianchi et al, 2019a).…”

“…In this paper, we use the LOCAL communication protocol (Suomela, 2013;Fraigniaud, 2016;Linial, 1992), which has recently seen an increase in interest in the decentralized multi-agent bandit literature (Cesa-Bianchi et al, 2019a;.…”

An Efficient Algorithm for Cooperative Semi-Bandits

“…Finally, the reader may wonder what kind of results could be achieved if the agents are activated adversarially rather than stochastically. Cesa-Bianchi et al [2019a] showed that in this setting no learning can occur, not even in with full-information feedback.…”

“…In our multi-agent setting, the end goal is to control the total network regret (1). This objective was already studied by Cesa-Bianchi et al [2019a] in the full-information case. A similar line of work was pursued by Cesa-Bianchi et al [2019b], where the authors consider networks of learning agents that cooperate to solve the same nonstochastic bandit problem.…”