Distributed Reinforcement Learning based MAC protocols for autonomous cognitive secondary users

Bkassiny, Mario; Jayaweera, Sudharman K.; Avery, Keith

doi:10.1109/wocc.2011.5872298

Cited by 33 publications

(30 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, as a typical reinforcement learning framework, DQN has been applied in [7], [30], [31] for different purposes such as to improve the accuracy of selecting the channels in good condition, to maximize the network utility, or to minimize the service blocking probability. Additionally, in order to solve the dynamic spectrum access problem in decentralized systems, different multi-agent reinforcement learning strategies are studied in DRAFT [30], [32], [33]. For instance, in [30], the authors concentrated on a multi-user scenario in which transmission is successful only if a single user transmits over an accessed channel.…”

Section: Related Workmentioning

confidence: 99%

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

Zhong

Gursoy

et al. 2019

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework.Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.

show abstract

Section: Related Workmentioning

confidence: 99%

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

Zhong

Gursoy

et al. 2019

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…Through CRs techniques of dynamic spectrum sharing (DSS) are developed for efficiently using crowded frequency bands. [21,30,31,32] In recent years research has been done on applying machine learning techniques on CRs [33], [34].…”

Section: Cognitive Radio and Annmentioning

confidence: 99%

A Survey on Cognitive Radio Network using Artificial Neural Network

Waqar

Khadim²,

Zeb³

et al. 2017

IJFGCN

View full text Add to dashboard Cite

show abstract

“…Based on (18), the convolution of S 0 x (f ) with a window of length 2f max causes the PSD to spread at most by ±f max at each point. If the Doppler PSD S μμ (f ) is symmetric (such as Jakes' type [25]), the carrier frequency components of the detected feature points do not shift since the main lobes of the PSD are spread evenly in both left and right directions.…”

Section: Impact Of the Doppler Shift On The Detected Carrier Freqmentioning

confidence: 99%

“…Several learning algorithms have been previously applied to CR's for PHY/MAC decision-making. In particular, the reinforcement learning (RL) has been applied for power control [16] and for distributed Medium Access Control (MAC) in CR networks [17], [18]. In our case, however, the Radiobot employs a learning algorithm similar to [19], allowing online self-reconfiguration of the spectrum sensing module.…”

Section: Introductionmentioning

confidence: 99%

Wideband Spectrum Sensing and Non-Parametric Signal Classification for Autonomous Self-Learning Cognitive Radios

Bkassiny

Jayaweera

et al. 2012

IEEE Trans. Wireless Commun.

Self Cite

View full text Add to dashboard Cite

This paper presents an autonomous cognitive radio (CR) architecture, referred to as the Radiobot. This model goes beyond adaptive radio systems to exploit the main ingredients of cognition which, in this context, are mainly self-learning and self-reconfiguration. Without any prior knowledge of the RF environment, the Radiobot applies a sequence of increasingly sophisticated processing steps to detect and identify the sensed signals. In particular, in this paper, it applies a blind energy detection followed by a cyclostationary detection method to detect the active signals and extract their underlying periodic properties as reflected in cyclic frequencies. These extracted signal features are classified based on the Chinese restaurant process (CRP) and a learning algorithm is applied to achieve autonomous selfreconfiguration of the sensing module. We analyze the impact of fading and Doppler frequency shift on both the energy and cyclostationary detections, and show the receiver operating characteristic (ROC) of the carrier frequency detector. We show the robustness of the cyclostationary detection against channel fading and wide-sense stationary noise. Simulation results are presented to verify the multi-band operability and the reconfiguration ability of the Radiobot and to verify the convergence of the proposed learning algorithm.

show abstract

Distributed Reinforcement Learning based MAC protocols for autonomous cognitive secondary users

Cited by 33 publications

References 14 publications

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

A Survey on Cognitive Radio Network using Artificial Neural Network

Wideband Spectrum Sensing and Non-Parametric Signal Classification for Autonomous Self-Learning Cognitive Radios

Contact Info

Product

Resources

About