Sleeping Multi-Armed Bandits for Fast Uplink Grant Allocation in Machine Type Communications

Abstract

Scheduling fast uplink grant transmissions for machine type communications (MTCs) is one of the main challenges of future wireless systems. In this paper, a novel fast uplink grant scheduling method based on the theory of multi-armed bandits (MABs) is proposed. First, a single quality-of-service metric is defined as a combination of the value of data packets, maximum tolerable access delay, and data rate. Since full knowledge of these metrics for all machine type devices (MTDs) cannot be known in advance at the base station (BS) and the set of active MTDs changes over time, the problem is modeled as a sleeping MAB with stochastic availability and a stochastic reward function. In particular, given that at each time step, the knowledge on the set of active MTDs is probabilistic, a novel probabilistic sleeping MAB algorithm is proposed to maximize the defined metric. Numerical results show that the proposed algorithm has logarithmic regret, and hence is optimal. The results also show that the proposed framework yields a three-fold reduction in latency compared to a random scheduling policy since it prioritizes the scheduling of MTDs that have stricter latency requirements. Moreover, by properly balancing the exploration versus exploitation tradeoff, the proposed algorithm is able to provide system fairness by allowing the most important MTDs to be scheduled more often while also allowing the less important MTDs to be selected enough times to ensure the accuracy of estimation of their importance.

Traffic Prediction and Fast Uplink for Hidden Markov IoT Models

“…Sinusoidal spreading sequences were proposed in [21] to enable FU grant based on free nonorthogonal multiple access (NOMA), whereas authors in [22] discussed hybrid resource allocation schemes to overcome the large signaling overhead and collision problems resulting from message replications in GF transmission. Moreover, in [23], Samad et al introduced a multi-armed bandit algortihm to perform FU grant in IoT networks. However, this work also came short from exploiting the traffic correlation on the eventtemporal basis.…”

“…. by the random variable GF-RA S. M. Hasan et al [21] NOMA FU grant Z. Zhou et al [22] Hybrid resource allocation FU grant S. Ali et al [23] Multi-armed bandit E. Eldeeb et al [24] SVM and LSTM O. Habachi et al [25] Federated learning Deep learning I. AlQerm et al [26] Reinforcement learning A. E. Kalør et al [27] Meta-learning and RNN F. Mohammadi et al [28] Multi-agent DRL X. Liu et al [29] Clustering D. Hejji et al [30] AI survey…”

Energy-efficient online resource provisioning for cloud-edge platforms via multi-armed bandits

“…However, they refer to factors such as the transmitted power level or the radio access technology, whereas our work focuses on hardware factors such as CPU and/or memory availability. Ali S. et al [6] present a fast uplink grant scheduling method based on a probabilistic sleeping MABs (i.e. the set of available arms varies over time) for machine type devices (MTDs) in wireless systems.…”

Traffic Prediction Based Fast Uplink Grant for Massive IoT

“…The FU approach suggests that the aggregator is able to predict the likelihood of the traffic pattern of the 4 devices and grants the 2 available transmission slots to the 2 devices which are more likely to transmit. In [7], Samad et. al proposed a multi-armed bandit framework to perform fast uplink grant for IoT devices.…”