Distributed Bandits with Heterogeneous Agents

Yang, Lin; Chen, Yu-zhen Janice; Hajiesmaili, Mohammad H.; Lui, John CS; Towsley, Don

doi:10.1109/infocom48880.2022.9796901

Cited by 6 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As done in the classic work on regret minimization in single-agent bandits [2,12,50], our goal is to minimize the group regret, which has been studied widely in the context of collaborative multi-agent bandits [30,42,61]. As the name suggests, the agents can (and should) collaborate with each other over some given communication network to minimize the group regret, defined as…”

Section: Goalsmentioning

confidence: 99%

“…Here, we define communication complexity as the total number of message exchanges, with the message consisting of arm index, observed reward, and possibly other information. In homogeneous settings, where all agents are identical in that K 𝑣 = K for all 𝑣 ∈ V, arm elimination-type algorithms [61] as well as gossip-type protocols [16,51] have been shown to be communication efficient and effective in terms of group regret.…”

Section: Goalsmentioning

confidence: 99%

“…Another common assumption in the literature is that agents share the same set of arms. However, this may not hold in many large-scale systems, where often, agents have heterogeneous sets of available actions [61]. While there is a variety of literature on agents communicating over a graph while sharing the same set of arms [26,29,30,43,44,46], as well as works analyzing the dynamics exhibited by arm-or reward-heterogeneous agents on a complete graph [7,17,61,62], the setting of agents that are heterogeneous both in their network topology as well as their action set, to the best of our knowledge, has not been investigated yet.…”

Section: Introductionmentioning

confidence: 99%

“…However, this may not hold in many large-scale systems, where often, agents have heterogeneous sets of available actions [61]. While there is a variety of literature on agents communicating over a graph while sharing the same set of arms [26,29,30,43,44,46], as well as works analyzing the dynamics exhibited by arm-or reward-heterogeneous agents on a complete graph [7,17,61,62], the setting of agents that are heterogeneous both in their network topology as well as their action set, to the best of our knowledge, has not been investigated yet. Such a setting in fact arises in a wide range of real-life scenarios, such as location-dependent online advertisements, social networks, and recommender systems, where collaboration between different instances more efficiently generates user preference profiles [61].…”

Section: Introductionmentioning

confidence: 99%

“…While there is a variety of literature on agents communicating over a graph while sharing the same set of arms [26,29,30,43,44,46], as well as works analyzing the dynamics exhibited by arm-or reward-heterogeneous agents on a complete graph [7,17,61,62], the setting of agents that are heterogeneous both in their network topology as well as their action set, to the best of our knowledge, has not been investigated yet. Such a setting in fact arises in a wide range of real-life scenarios, such as location-dependent online advertisements, social networks, and recommender systems, where collaboration between different instances more efficiently generates user preference profiles [61]. For example, a network of content providers with very similar sets of users may want to maximize click rates for their (partially overlapping) content; in this case, arms correspond to the content shown to users, and rewards correspond to user opinions.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Communication-Efficient Collaborative Heterogeneous Bandits in Networks

Lee¹,

Schmid²,

Yun³

2023

Preprint

View full text Add to dashboard Cite

The multi-agent multi-armed bandit problem has been studied extensively due to its ubiquity in many real-life applications, such as online recommendation systems and wireless networking. We consider the setting where agents should minimize their group regret while collaborating over a given graph via some communication protocol and where each agent is given a different set of arms. Previous literature on this problem only considered one of the two desired features separately: agents with the same arm set communicate over a general graph, or agents with different arm sets communicate over a fully connected graph. In this work, we introduce a more general problem setting that encompasses all the desired features. For this novel setting, we first provide a rigorous regret analysis for the standard flooding protocol combined with the UCB policy. Then, to mitigate the issue of high communication costs incurred by flooding, we propose a new protocol called Flooding with Absorption (FWA). We provide a theoretical analysis of the regret bound and intuitions on the advantages of using FWA over flooding. Lastly, we verify empirically that using FWA leads to significantly lower communication costs despite minimal regret performance loss compared to flooding. CCS CONCEPTS• Theory of computation → Multi-agent learning; Online learning theory; Sequential decision making; Regret bounds; Distributed algorithms; • Networks → Network protocol design.

show abstract