Multi-Player Bandits Revisited

Abstract

Multi-player Multi-Armed Bandits (MAB) have been extensively studied in the literature, motivated by applications to Cognitive Radio systems. Driven by such applications as well, we motivate the introduction of several levels of feedback for multi-player MAB algorithms. Most existing work assume that sensing information is available to the algorithm. Under this assumption, we improve the state-of-theart lower bound for the regret of any decentralized algorithms and introduce two algorithms, RandTopM and MCTopM, that are shown to empirically outperform existing algorithms. Moreover, we provide strong theoretical guarantees for these algorithms, including a notion of asymptotic optimality in terms of the number of selections of bad arms. We then introduce a promising heuristic, called Selfish, that can operate without sensing information, which is crucial for emerging applications to Internet of Things networks. We investigate the empirical performance of this algorithm and provide some first theoretical elements for the understanding of its behavior.

Dominate or Delete: Decentralized Competing Bandits in Serial Dictatorship

“…In our system, as the agents have uniform rank across all arms there exists a unique stable matching (which is not true when agents are ranked non-uniformly across arms.) Indeed, in any stable match agent j must match with it's most preferred arm which is not matched with any agent with rank (j − 1) or higher 5 .…”

“…Thus, agents cannot infer if its actions cause a collision to other higher The arm-means for sub-optimal arms for each agent are chosen i.i.d. uniformly over [0, 0.8], while the arm-mean of i ∈ [5] for agent i was set to 0.9. The rewards are binary.…”

“…Figure3: Simulations on a system with 5 agents and 7 arms. For each agent i ∈[5], a permutation over the arms σ i was chosen, and the arm-means are equally spaced among the 7 arms from 0.1 to 0.9 in the increasing order of permutation. This is thus not a OSB instance.…”

“…This is done for convenience. Our analysis can be easily adapted to any sub-gaussian reward 5. Agents ranked 1 through j − 1…”

Decentralized Learning for Channel Allocation in IoT Networks over Unlicensed Bandwidth as a Contextual Multi-player Multi-armed Bandit Game

“…Due to decentralized and simultaneous policy selection, collisions have to be taken into consideration when more than one player happen to choose the same arm. For this situation, a number of studies [21]- [23] assume that no player receives any reward, while some other studies [20] assume that the colliding players split the reward over the single arm in an arbitrary way. Such a model is frequently used to describe the user-channel matching problem in a CRN, where the channel condition is modeled to be stochastic, partially due to the unpredictable activities of the primary users [16], [25].…”