Spectrum Access In Cognitive Radio Using a Two-Stage Reinforcement Learning Approach

Raj, Vishnu; Dias, Irene; Tholeti, Thulasi; Kalyani, Sheetal

doi:10.1109/jstsp.2018.2798920

Cited by 93 publications

(34 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[15] studied the energy-efficiency in a HetNet, and proposed a RL-based user scheduling and resource allocation algorithm. [16] and [17] investigated the spectrum sharing problem in cognitive radio networks and developed RL-based spectrum access algorithms for cognitive users. [18] and [19] focused on the self-organization network and adopted RL to deal with the request coordination problem and the user scheduling problem, respectively.…”

Section: B Related Workmentioning

confidence: 99%

Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

Zhang

Tan

Liang

et al. 2019

IEEE Trans. Wireless Commun.

126

View full text Add to dashboard Cite

We consider a cognitive heterogeneous network (HetNet), in which multiple pairs of secondary users adopt sensing-based approaches to coexist with a pair of primary users on a certain spectrum band. Due to imperfect spectrum sensing, secondary transmitters (STs) may cause interference to the primary receiver (PR) and make it difficult for the PR to select a proper modulation and/or coding scheme (MCS). To deal with this issue, we exploit deep reinforcement learning (DRL) and propose an intelligent MCS selection algorithm for the primary transmission. To reduce the system overhead caused by MCS switchings, we further introduce a switching cost factor in the proposed algorithm. Simulation results show that the primary transmission rate of the proposed algorithm without the switching cost factor is 90% ∼ 100% of the optimal MCS selection scheme, which assumes that the interference from the STs is perfectly known at the PR as prior information, and is 30% ∼ 100% higher than those of the benchmark algorithms. Meanwhile, the proposed algorithm with the switching cost factor can achieve a better balance between the primary transmission rate and system overheads than both the optimal algorithm and benchmark algorithms.

show abstract

Section: B Related Workmentioning

confidence: 99%

Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

Zhang

Tan

Liang

et al. 2019

IEEE Trans. Wireless Commun.

126

View full text Add to dashboard Cite

show abstract

“…(To know more about energy harvesting and related technologies, refer to [190]). In [191], the authors proposed a two-stage learning algorithm to reduce the channel sensing period. They used RL and the Bayesian method for learning.…”

Section: ) Spectrum Sensingmentioning

confidence: 99%

Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks

et al. 2020

View full text Add to dashboard Cite

Nowadays, machine learning (ML), which is one of the most rapidly growing technical tools, is extensively used to solve critical challenges in various domains. Vehicular ad hoc network (VANET) is expected to be the key role player in reducing road casualties and traffic congestion. To ensure this role, a gigantic amount of data should be exchanged. However, current allocated wireless access for VANET is inadequate to handle such massive data amounts. Therefore, VANET faces a spectrum scarcity issue. Cognitive radio (CR) is a promising solution to overcome such an issue. CR-based VANET or CR-VANET must achieve several performance enhancement measures, including ultra-reliable and lowlatency communication. ML methods can be integrated with CR-VANET to make CR-VANET highly intelligent, achieve rapid adaptability to the dynamicity of the environment, and improve the quality of service in an energy-efficient manner. This paper presents an overview of ML, CR, VANET, and CR-VANET, including their architectures, functions, challenges, and open issues. The applications and roles of ML methods in CR-VANET scenarios are reviewed. Insights into the use of ML for autonomous or driverless vehicles are also presented. Current advancements in the amalgamation of these prominent technologies and future research directions are discussed. INDEX TERMS Machine learning, VANET, cognitive radio, autonomous vehicles, smart transportation system. performance of CR-VANET [15]. Security enhancement is one of the major issues in CR-VANET. Here, a vehicle can pretend to be a PU and propagate false information to obtain spectrum access selfishly. ML can be used to detect such actions and enhance security [16], [17]. ML also provides an optimum route to CR-VANET users to avoid traffic jams and road accidents. ML can also play a vital role in the best infotainment experience in CR-VANET. It can be used for appropriate scheduling, selecting the best channel, and prioritizing messages. CR and ML can play a major role in the next-generation driverless car system. The role of CR in the next-generation transportation system has been presented in previous discussions. This survey shows how ML can be applied to reduce road accidents and traffic congestion. CR can be used to accommodate the spectrum required to support massive data communication among automated driverless vehicles and networks. ML can be an integral part of this driverless or automated vehicle system. Similar to a robot, an autonomous vehicle (AV) can learn the surrounding environment and communicate with increased safety, reliability, QoS, and energy efficiency by applying such learning. This paper presents the dynamic usages of ML in CR-VANET elaborately. Several of the benefits of CR in VANETs and ML in VANETs and CR-VANETs are presented in Table 1. B. CONTRIBUTIONS OF THIS SURVEY ARTICLE Many survey articles describe CR, VANET, ML, and CR-VANETs individually or describe a few aspects of their amalgamation. To the best of our knowledge, surveys that cover the usage of ML in CR-VANET...

show abstract

“…The authors in [23] proposed various applications of RL algorithms in CRNs. Conventional interweave CRNs have been studied in [24]- [29] where the authors proposed RL based spectrum sensing policies. In [30], the authors considered an overlay EH-CRN where the EH-SU helped EH-PU deliver its data.…”

Section: A Background Workmentioning

confidence: 99%

Harvest-or-Transmit Policy for Cognitive Radio Networks: A Learning Theoretic Approach

Pathak

Banerjee

2019

IEEE Trans. on Green Commun. Netw.

View full text Add to dashboard Cite

We consider an underlay cognitive radio network where the secondary user (SU) harvests energy from the environment. We consider a slotted-mode of operation where each slot of SU is used for either energy harvesting or data transmission. Considering block fading with memory, we model the energy arrival and fading processes as a stationary Markov process of first order. We propose a harvest-or-transmit policy for the SU along with optimal transmit powers that maximize its expected throughput under three different settings. First, we consider a learning-theoretic approach where we do not assume any apriori knowledge about the underlying Markov processes. In this case, we obtain an online policy using Q-learning. Then, we assume that the full statistical knowledge of the governing Markov process is known apriori. Under this assumption, we obtain an optimal online policy using infinite horizon stochastic dynamic programming. Finally, we obtain an optimal offline policy using the generalized Benders decomposition algorithm. The offline policy assumes that for a given time deadline, the energy arrivals and channel states are known in advance at all the transmitters. Finally, we compare all policies and study the effects of various system parameters on the system performance.

show abstract

Spectrum Access In Cognitive Radio Using a Two-Stage Reinforcement Learning Approach

Cited by 93 publications

References 35 publications

Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks

Harvest-or-Transmit Policy for Cognitive Radio Networks: A Learning Theoretic Approach

Contact Info

Product

Resources

About