Decentralized Learning for Channel Allocation in IoT Networks over Unlicensed Bandwidth as a Contextual Multi-player Multi-armed Bandit Game

Abstract

We study a decentralized channel allocation problem in an ad-hoc Internet of Things (IoT) network underlaying on a spectrum licensed to an existing wireless network. In the considered IoT network, the impoverished computation capability and the limited antenna number on the IoT devices make them difficult to acquire the Channel State Information (CSI) for the multi-channels over the shared spectrum. In addition, in practice, the unknown patterns of the licensed users' transmission activities and the time-varying CSI due to fast fading or mobility of the IoT devices can also cause stochastic changes in the channel quality. Therefore, decentralized IoT links are expected to learn their channel statistics online based on the partial observations, while acquiring no information about the channels that they are not operating on. Meanwhile, they also have to reach an efficient, collision-free solution of channel allocation on the basis of limited coordination or message exchange. Our study maps this problem into a contextual multi-player, multi-arm bandit game, for which we propose a purely decentralized, three-stage policy learning algorithm through trial-and-error. Our theoretical analysis shows that the proposed learning algorithm guarantees the IoT devices to jointly converge to the social-optimal channel allocation with a sub-linear (i.e., polylogarithmic) regret with respect to the operational time. Simulation results demonstrate that the proposed algorithm strikes a good balance between efficient channel allocation and network scalability when compared with the other state-of-the-art distributed multiarmed bandit algorithms.

Medium Access Control protocol for Collaborative Spectrum Learning in Wireless Networks

“…Therefore, to approach optimal utilization, approaches based on random access are used to converge to an optimal orthogonal allocation. Examples for such approaches utilizing Aloha are [18], [19], [20] and [21]. Similarly, CSMA based solution have also beem suggested, e.g.…”

“…These variants aim to achieve orthogonal allocations in a fully-distributed manner without any information sharing between nodes. Some of these variants aim to achieve optimal OFDMA allocations where the sum of the Quality of Service (QoS) of all the users is maximized [18], [19], [20], [25], [26], [27] while others aim to achieve stable OFDMA allocations where no user can unilaterally improve its QoS [22], [23], [24], [29]. Recently, new approaches incorporating fairness have also been proposed [21], [30].…”

“…This algorithm was inspired by a Perturbed Markov Chain Approach from [42] which solves the distributed optimization for a known game. A related approach was presented in [20] for IoT networks where contextual bandit model has been proposed. Min-Max fairness was achieved in [21], [30] while maintaining a near-optimal regret bound of O(ln T ln ln T ).…”

“…2) Carrier sensing based methods: Whereas collision based multi-access protocols such as Aloha and its advanced variants [19], [20] are appropriate for simple devices such as sensor networks they are inefficient for wireless communication adhoc network since convergence time to the optimum can be significant. In particular, the convergence time of Markov chain based method may be exponential in the number of users and channels.…”