Distributed Learning and Optimal Assignment in Multiplayer Heterogeneous Networks

Tibrewal, Harshvardhan; Patchala, Sravan; Hanawal, Manjesh K.; Darak, Sumit J.

doi:10.1109/infocom.2019.8737653

Cited by 35 publications

(20 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithms proposed in [6] and [25] are able to achieve a near-O(log T ) expected regret using the doubling trick and O(log T ) expected regret in the one-shot scenario (when T is known). Authors in [27] provide a distributed algorithm that aims to achieve optimal network throughput without any direct communication among players. However, the algorithm has a binary signalling phase that allows players to exchange information by transmitting in specific patterns, and players are also allowed to sense such transmissions from other players.…”

Section: Related Workmentioning

confidence: 99%

Decentralized Dynamic Rate and Channel Selection Over a Shared Spectrum

Javanmardi

Qureshi

Tekin

2021

IEEE Trans. Commun.

View full text Add to dashboard Cite

We consider the problem of distributed dynamic rate and channel selection in a multi-user network, in which each user selects a wireless channel and a modulation and coding scheme (corresponds to a transmission rate) in order to maximize the network throughput. We assume that the users are cooperative, however, there is no coordination and communication among them, and the number of users in the system is unknown. We formulate this problem as a multi-player multi-armed bandit problem and propose a decentralized learning algorithm that performs almost optimal exploration of the transmission rates to learn fast. We prove that the regret of our learning algorithm with respect to the optimal allocation increases logarithmically over rounds with a leading term that is logarithmic in the number of transmission rates. Finally, we compare the performance of our learning algorithm with the state-of-the-art via simulations and show that it substantially improves the throughput and minimizes the number of collisions.

show abstract

Section: Related Workmentioning

confidence: 99%

Decentralized Dynamic Rate and Channel Selection Over a Shared Spectrum

Javanmardi

Qureshi

Tekin

2021

IEEE Trans. Commun.

View full text Add to dashboard Cite

show abstract

“…For instance, Game of Thrones (GoT) in [31] is a fully distributed algorithm that solves the distributed assignment problem via collision and reward feedbacks. In [32], random exploration in GoT is replaced with channel orthogonalization in order to improve the reward estimation in exploration phase. Contrary to GoT, where collisions result in zero reward, authors in [33] consider the case in which colliding users receive non-zero rewards.…”

Section: B Comparison With Related Workmentioning

confidence: 99%

Multi-User Small Base Station Association via Contextual Combinatorial Volatile Bandits

Qureshi

Nika

Tekin

2021

IEEE Trans. Commun.

View full text Add to dashboard Cite

We propose an efficient mobility management solution to the problem of assigning small base stations (SBSs) to multiple mobile data users in a heterogeneous setting. We formalize the problem using a novel sequential decision-making model named contextual combinatorial volatile multi-armed bandits (MABs), in which each association is considered as an arm, volatility of an arm is imposed by the dynamic arrivals of the users, and context is the additional information linked with the user and the SBS such as user/SBS distance and the transmission frequency. As the next-generation communications are envisioned to take place over highly dynamic links such as the millimeter wave (mmWave) frequency band, we consider the association problem over an unknown channel distribution with a limited feedback in the form of acknowledgments and under the absence of channel state information (CSI). As the links are unknown and dynamically varying, the assignment problem cannot be solved offline. Thus, we propose an online algorithm which is able to solve the user-SBS association problem in a multi-user and time-varying environment, where the number of users dynamically varies over time. Our algorithm strikes the balance between exploration and exploitation and achieves sublinear in time regret with an optimal dependence on the problem structure and the dynamics of user arrivals and departures. In addition, we demonstrate via numerical experiments that our algorithm achieves significant performance gains compared to several benchmark algorithms.

show abstract

“…On the other hand, the works Landgren et al [2016aLandgren et al [ ,b, 2021, Martínez-Rubio et al [2019], Dubey and Pentland [2020] restricts the communication over the network, wherein agents can only communicate with their neighbors encoded by an undirected communication graph with an unknown structure. Another branch of works Bistritz et al [2020], Kalathil et al [2014, Liu et al [2013], Tibrewal et al [2019], Rosenski et al [2016], Bistritz and Leshem [2018] investigates a competitive setting where the reward becomes zero or the reward is split when two or more agents select the same arm simultaneously. These competitive models are considered for some real-world applications wherein the communication between agents is almost impossible, and sometimes agents need to learn from the collision rather than communicating information.…”

Section: Related Workmentioning

confidence: 99%

Cooperative Stochastic Multi-agent Multi-armed Bandits Robust to Adversarial Corruptions

Liu,

Li,

2021

Preprint

View full text Add to dashboard Cite

We study the problem of stochastic bandits with adversarial corruptions in the cooperative multi-agent setting, where V agents interact with a common K-armed bandit problem, and each pair of agents can communicate with each other to expedite the learning process. In the problem, the rewards are independently sampled from distributions across all agents and rounds, but they may be corrupted by an adversary. Our goal is to minimize both the overall regret and communication cost across all agents. We first show that an additive term of corruption is unavoidable for any algorithm in this problem. Then, we propose a new algorithm that is agnostic to the level of corruption. Our algorithm not only achieves near-optimal regret in the stochastic setting, but also obtains a regret with an additive term of corruption in the corrupted setting, while maintaining efficient communication. The algorithm is also applicable for the single-agent corruption problem, and achieves a high probability regret that removes the multiplicative dependence of K on corruption level. Our result of the single-agent case resolves an open question from Gupta et al. [2019].Preprint. Under review.

show abstract

Distributed Learning and Optimal Assignment in Multiplayer Heterogeneous Networks

Cited by 35 publications

References 21 publications

Decentralized Dynamic Rate and Channel Selection Over a Shared Spectrum

Decentralized Dynamic Rate and Channel Selection Over a Shared Spectrum

Multi-User Small Base Station Association via Contextual Combinatorial Volatile Bandits

Cooperative Stochastic Multi-agent Multi-armed Bandits Robust to Adversarial Corruptions

Contact Info

Product

Resources

About