Differentially-Private Federated Linear Bandits

Abstract

The rapid proliferation of decentralized learning systems mandates the need for differentially-private cooperative learning. In this paper, we study this in context of the contextual linear bandit: we consider a collection of agents cooperating to solve a common contextual bandit, while ensuring that their communication remains private. For this problem, we devise FEDUCB, a multiagent private algorithm for both centralized and decentralized (peer-to-peer) federated learning. We provide a rigorous technical analysis of its utility in terms of regret, improving several results in cooperative bandit learning, and provide rigorous privacy guarantees as well. Our algorithms provide competitive performance both in terms of pseudoregret bounds and empirical benchmark performance in various multi-agent settings.1 Originally, federated learning referred to the algorithm proposed in [28] for supervised learning, however, now the term broadly refers to the distributed cooperative learning setting [27].34th Conference on Neural Information Processing Systems (NeurIPS 2020),

Asynchronous Upper Confidence Bound Algorithms for Federated Linear Bandits

“…However, they only tried to reduce per-round communication, and thus the communication cost is still linear over time. Two follow-up studies considered the setting where all agents solve a common problem and interact with the environment in a round-robin fashion (Wang et al, 2019;Dubey and Pentland, 2020). Similar to our work, they also used event-triggered communications to obtain a sub-linear communication cost over time.…”

“…Distributed bandit (Korda et al, 2016;Wang et al, 2019;Dubey and Pentland, 2020) is the most relevant to ours, where designing an efficient communication strategy is the main focus. Existing algorithms mainly differ in the relations of learning problems solved by the agents (i.e., identical vs., clustered) and the type of communication network (i.e., peer-to-peer (P2P) vs., star-shaped).…”

“…In particular, Wang et al (2019) considered a star-shaped network and proposed a synchronous communication protocol for all clients to exchange their latest observations under the central server's control. Dubey and Pentland (2020) extended this synchronous protocol to differentially private LinUCB algorithms under both star-shaped and P2P network.…”

“…Note that we do not impose any further assumption on the clients' distribution or frequency of its interactions. This makes our setting more general than existing works (Wang et al, 2019;Dubey and Pentland, 2020), where the clients interact with the environment in a round-robin fashion, i.e., |N i (t)| = t/N . In addition, in the federated learning setting, the clients cannot directly communicate with each other, but can only communicate with the central server, i.e., a star-shaped communication network.…”

“…In addition, the clients often have various response time and even occasional unavailability in reality, due to the differences in their computational and communication capacities. This hampers global synchronization employed in existing federated bandit solutions (Wang et al, 2019;Dubey and Pentland, 2020), which requires the server to first send a synchronization signal to all clients, wait and collect their returned local updates, and finally send the aggregated update back to every client. Second, it is very restrictive to only assume homogeneous clients, i.e., they solve the same learning problem.…”