Enabling opportunistic and dynamic spectrum access through learning techniques

Abstract

The expected shortage in spectrum supply is well understood to be primarily due to the inefficient, static nature of current spectrum allocation policies. In order to address this problem, Federal Communications Commission promotes the so called opportunistic spectrum access (OSA) to be applied on cognitive radio networks (CRNs). In short, the idea behind OSA is allowing unlicensed users to use unused licensed spectra as long as they do not cause interference to licensed users. In this paper, we present and evaluate learning schemes that allow unlicensed users to locate and use spectrum opportunities effectively, thus improving efficiency of CRNs. We separately consider two models: single and multiple unlicensed user(s). For the latter model, we present two schemes: noncooperative and cooperative Q-learning. All proposed schemes do not require prior knowledge or prediction models of the environment's dynamics and behaviors, yet can still achieve high performance by learning from interaction with the environment. Using simulations, we show that the proposed schemes achieve good performances in terms of throughput and fairness.

Reinforcement Learning with Budget-Constrained Nonparametric Function Approximation for Opportunistic Spectrum Access

“…[22][23][24], and [19]. The majority of existing approaches consider finite state and action spaces where the Q-function is stored in a table [19,22,23], whereas we consider continuous state spaces that are handled using kernel based function approximation. RL with function approximation is considered in [24,25].…”

The impact of stochastic resource availability on cognitive network performance: modeling and analysis

“…Because of its great potentials in addressing the spectrum shortage problem, the cognitive radio network paradigm has attracted significant research focus over the past decade, addressing various different aspects, such as protocol design , spectrum sensing , resource allocation and management , performance modeling and analysis , and spectrum trading and auction . Delay performance analysis has also received some attention, but not as much .…”

Improving quality of data user experience in 4G distributed telecommunication systems

“…This environment gives rise to unique characteristics, which make it too difficult for users to model/predict its dynamics and behaviors [3][4][5][6][7]. As such, learning-based techniques that do not require prediction models, but can still manage well the network resources by learning through their interactions with the environment are particularly well suited to this type of environment, whose behavior is, by nature, too complex to predict, but the QoE to be achieved as a result of using the environment can easily be assessed/observed [8][9][10][11][12][13][14]. Instead of using prediction models, these techniques rely on learning algorithms, such as reinforcement learners [15,16], to learn from past and present interaction experience to decide what to do best in the future.…”