Decentralized Dynamic Rate and Channel Selection Over a Shared Spectrum

Abstract

We consider the problem of distributed dynamic rate and channel selection in a multi-user network, in which each user selects a wireless channel and a modulation and coding scheme (corresponds to a transmission rate) in order to maximize the network throughput. We assume that the users are cooperative, however, there is no coordination and communication among them, and the number of users in the system is unknown. We formulate this problem as a multi-player multi-armed bandit problem and propose a decentralized learning algorithm that performs almost optimal exploration of the transmission rates to learn fast. We prove that the regret of our learning algorithm with respect to the optimal allocation increases logarithmically over rounds with a leading term that is logarithmic in the number of transmission rates. Finally, we compare the performance of our learning algorithm with the state-of-the-art via simulations and show that it substantially improves the throughput and minimizes the number of collisions.

Distributed Learning Over Markovian Fading Channels for Stable Spectrum Access

“…In [10], the authors reduced the communication burden, but without guarantees on the achievable regret. Recently, it was shown in [12]- [15] that achieving a sum-regret of near-O(log t) is possible without communication between users, but only for the case of i.i.d or rested Markovian channels. In this paper, we focus on the general case where the channel states may change whether or not they are being observed (i.e., the restless Markovian setting) and improve the regret scaling with the system parameters by a simple distributed implementation.…”

“…a: A general model for spectrum access using a restless Markovian channel model As explained above, by contrast to [6]- [8], [10], [12]- [15], in this paper, we first solve the channel allocation and access problem under general unknown restless Markovian channel model. The restless model is capable of capturing more realistic and complex scenarios, and handling this model adds significant challenges in algorithm design and regret analysis.…”

“…When comparing to existing approaches, DSSL achieves this under the more general restless Markovian model, and also has significantly better scaling with the system parameters. Specifically, under a common benchmark setting of equal rates among users (but still vary among channels), and K > M , which allows a theoretical comparison of learning efficiency between different algorithms, in [8], [15] and [16] the regret scales as O( M K (∆min) 2 log(t)), in [14] as O( M 3 K (∆min) 2 log(t)) and in [11] the regret scales as…”

“…In the more general problem considered in this paper, we show in the Appendix that the value of L depends on the parameters of the Markov process which determine its rate of convergence to a steady state. The exploration rate depends on L as derived in (15). The expectations µ i,k are given by:…”

“…Send an interrupt signal in the Appendix shows that exploring channels as in (15) guarantees the desired convergence speed. Let M i,k x,y be the mean hitting time of state y starting at initial state x for channel k used by user i, and…”