Learning Multiuser Channel Allocations in Cognitive Radio Networks: A Combinatorial Multi-Armed Bandit Formulation

Abstract

Abstract-We consider the following fundamental problem in the context of channelized dynamic spectrum access. There are M secondary users and N ≥ M orthogonal channels. Each secondary user requires a single channel for operation that does not conflict with the channels assigned to the other users. Due to geographic dispersion, each secondary user can potentially see different primary user occupancy behavior on each channel. Time is divided into discrete decision rounds. The throughput obtainable from spectrum opportunities on each userchannel combination over a decision period is modeled as an arbitrarily-distributed random variable with bounded support but unknown mean, i.i.d. over time. The objective is to search for an allocation of channels for all users that maximizes the expected sum throughput. We formulate this problem as a combinatorial multi-armed bandit (MAB), in which each arm corresponds to a matching of the users to channels. Unlike most prior work on multi-armed bandits, this combinatorial formulation results in dependent arms. Moreover, the number of arms grows super-exponentially as the permutation P (N, M ). We present a novel matching-learning algorithm with polynomial storage and polynomial computation per decision period for this problem, and prove that it results in a regret (the gap between the expected sum-throughput obtained by a genie-aided perfect allocation and that obtained by this algorithm) that is uniformly upper-bounded for all time n by a function that grows as O(M 4 N logn), i.e. polynomial in the number of unknown parameters and logarithmic in time. We also discuss how our results provide a non-trivial generalization of known theoretical results on multi-armed bandits.

Sequential Learning for Multi-Channel Wireless Network Monitoring With Channel Switching Costs

“…In addition to learning the channel availability, the second users also learn the other users' strategies and the number of total users in the system through channel feedback. Existing work applying MAB in the cognitive radio context assumes identical channel view with the exception of Gai et al [23]. However, the model considered in this work, in fact makes the implicit assumption that all secondary users are colocated ("if there are multiple users on the channel, then we assume that, due to interference, at most one of the conflicting users gets reward").…”

“…However, the model considered in this work, in fact makes the implicit assumption that all secondary users are colocated ("if there are multiple users on the channel, then we assume that, due to interference, at most one of the conflicting users gets reward"). Since co-located secondary users likely observe identical primary user activities, a contradiction arises to the claim of "allowing the reward process on the same channel to be different" [23]. In our earlier work [24], we proposed two order-optimal policies for the channel assignment problem without switching costs.…”

Distributed and Centralized Schemes for Channel Sensing Order Setting in Multi-user Cognitive Radio Networks

“…And we focus on the simplest complete-interference case when all secondary users interfere with each other and must therefore each be allocated distinct channels. The same assumption is also adopted in [7] and [20], and is discussed in [23]. We use a M × N matrix P I to denote the primary free probabilities where its element P m,i is the primary free probability of channel i for user m, and P m,i is uncorrelated with P l,i for m = l; and a M × N matrix R to denote the channel achievable rates where its element R m,i is the achievable rate of channel i for user m, and R m,i is uncorrelated with R l,i for m = l. We assume that P I and R are known to the users a priori, the estimation of them is not our concern.…”

“…The optimal sensing order setting problem is formulated under the multi-armed bandit framework in [5][6][7], where a balance is found between longterm channel exploration and short-term channel exploitation to optimize the total throughput throughout the entire transmission of the SUs. However in this paper we focus on exploiting the instantaneous channel condition under the assumption that the statistics of channel availability and channel achievable rate are known a priori.…”

“…Channel sensing order is important for SUs to find a good available channel as fast as possible, and better channels and shorter sensing time will lead to higher throughput. Multi-armed bandit problems are used in [5][6][7] to solve the channel sensing ordering problems in uncertain environments, and polynomial regrets are obtained. In this paper, we assume that the primary users' activity and channels' achievable rates are known a priori to SUs.…”