Multi-Armed Bandits for Spectrum Allocation in Multi-Agent Channel Bonding WLANs

Abstract

While dynamic channel bonding (DCB) is proven to boost the capacity of wireless local area networks (WLANs) by adapting the bandwidth on a per-frame basis, its performance is tied to the primary and secondary channel selection. Unfortunately, in uncoordinated high-density deployments where multiple basic service sets (BSSs) may potentially overlap, hand-crafted spectrum management techniques perform poorly given the complex hidden/exposed nodes interactions. To cope with such challenging Wi-Fi environments, in this paper, we first identify machine learning (ML) approaches applicable to the problem at hand and justify why model-free RL suits it the most. We then design a complete RL framework and call into question whether the use of complex RL algorithms helps the quest for rapid learning in realistic scenarios. Through extensive simulations, we derive that stateless RL in the form of lightweight multi-armed-bandits (MABs) is an efficient solution for rapid adaptation avoiding the definition of broad and/or meaningless states. In contrast to most current trends, we envision lightweight MABs as an appropriate alternative to the cumbersome and slowly convergent methods such as Q-learning, and especially, deep reinforcement learning.

An Intelligent Optimization Method for Wireless Communication Network Resources Based on Reinforcement Learning

“…Most spectrum access studies assume that the number of users is less than the number of channels, which is somewhat different from reality. Barrachina-Muñoz et al [12] adopts a random dobby gambling machine model, assuming that the reward can be non-zero even in a collision. Therefore, it allows the number of users to exceed the number of channels.…”

Wi-Fi Meets ML: A Survey on Improving IEEE 802.11 Performance With Machine Learning

“…Then, dynamically adapting the different thresholds used to select one or another action based on the amplitude of the generated waveform at sampling instants shows that such a technique can outperform default MABs such as UCB and ε-greedy in terms of throughput. Finally, Barrachina-Muñoz et al [166] justify model-free RL techniques to address the channel bonding problem, design a complete RL framework and call into question whether complex RL algorithms allow rapid learning in realistic scenarios. Through extensive simulations, results show that a stateless RL in the form of lightweight MABs is an efficient solution for rapid adaptation, avoiding the definition of broad and/or meaningless states.…”

Federated spatial reuse optimization in next-generation decentralized IEEE 802.11 WLANS

“…ML, in this context, is considered a promising tool to capture the complex interactions between IEEE 802.11 devices applying SR. In general, ML has been applied to a plethora of problems in IEEE 802.11 networks, including PHY optimization (rate selection [22], resource allocation [23]), assisting management operations (e.g., AP selection and handover [24], channel band selection [25]), or supporting novel features like MU-MIMO or channel bonding with enhanced monitoring, analytics, and decision-making [26,27]. For further details on ML application to Wi-Fi, we refer the interested reader to the comprehensive survey in [28].…”