Contextual Bandit Learning for Machine Type Communications in the Null Space of Multi-Antenna Systems

Ali, Samad; Asgharimoghaddam, Hossein; Rajatheva, Nandana; Saad, Walid; Haapola, Jussi

doi:10.1109/tcomm.2019.2955454

Cited by 4 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Contextual bandits have been applied for fundamental areas in wireless communications [ 40 ] like machine type communications (MTC) [ 41 ], cooperative communications [ 42 ], link adaptation [ 43 ], and wireless handover optimization [ 44 ]. This motivates us to leverage CMAB to solve D2D NDS critical problem, especially with the challenging difficulties of mmWaves.…”

Section: Literature Reviewmentioning

confidence: 99%

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Hashima

Hatano

Kasban

et al. 2021

Sensors

View full text Add to dashboard Cite

The unique features of millimeter waves (mmWaves) motivate its leveraging to future, beyond-fifth-generation/sixth-generation (B5G/6G)-based device-to-device (D2D) communications. However, the neighborhood discovery and selection (NDS) problem still needs intelligent solutions due to the trade-off of investigating adjacent devices for the optimum device choice against the crucial beamform training (BT) overhead. In this paper, by making use of multiband (μW/mmWave) standard devices, the mmWave NDS problem is addressed using machine-learning-based contextual multi-armed bandit (CMAB) algorithms. This is done by leveraging the context information of Wi-Fi signal characteristics, i.e., received signal strength (RSS), mean, and variance, to further improve the NDS method. In this setup, the transmitting device acts as the player, the arms are the candidate mmWave D2D links between that device and its neighbors, while the reward is the average throughput. We examine the NDS’s primary trade-off and the impacts of the contextual information on the total performance. Furthermore, modified energy-aware linear upper confidence bound (EA-LinUCB) and contextual Thomson sampling (EA-CTS) algorithms are proposed to handle the problem through reflecting the nearby devices’ withstanding battery levels, which simulate real scenarios. Simulation results ensure the superior efficiency of the proposed algorithms over the single band (mmWave) energy-aware noncontextual MAB algorithms (EA-UCB and EA-TS) and traditional schemes regarding energy efficiency and average throughput with a reasonable convergence rate.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Hashima

Hatano

Kasban

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The contextual bandit is a special instance of the reinforcement learning model called multi-arm bandits (MABs) [13].…”

Section: B An Introduction To Deep Contextual Bandits (Dcb)mentioning

confidence: 99%

Deep Contextual Bandits for Fast Neighbor-Aided Initial Access in mmWave Cell-Free Networks

Ismath

Ali

Rajatheva

2021

IEEE Wireless Commun. Lett.

Self Cite

View full text Add to dashboard Cite

Access points (APs) in millimeter-wave (mmWave) user-centric (UC) networks will have sleep mode functionality. Initial access (IA) is a challenging problem in UC networks due to the coherent serving of the users. In this paper, a novel deep contextual bandit (DCB) learning-based instantaneous beam selection method is proposed as a complementary tool to current IA schemes. In the proposed approach, the DCB model at an AP uses beam selection information from the neighboring active APs as the input to solve the beam search problem of the host AP. The proposed fast beam selection scheme enables APs to be in energy-saving modes while maintaining the ability to serve users without any delay when restored. Simulations are carried out with realistic channel models generated using a ray-tracing tool. The results show that the proposed system with the 5G IA scheme can respond to dynamic throughput demands with negligible latency compared to the 5G IA scheme without the proposed scheme.

show abstract

Deep contextual bandits-based energy-efficient beamforming for integrated sensing and communication

Choukali,

Mirbolouk,

Valizadeh

et al. 2025

Physical Communication

View full text Add to dashboard Cite

Contextual Bandit Learning for Machine Type Communications in the Null Space of Multi-Antenna Systems

Cited by 4 publications

References 32 publications

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications

Deep Contextual Bandits for Fast Neighbor-Aided Initial Access in mmWave Cell-Free Networks

Deep contextual bandits-based energy-efficient beamforming for integrated sensing and communication

Contact Info

Product

Resources

About