A Markov Decision Process-Based Opportunistic Spectral Access

Arunthavanathan, Senthuran; Kandeepan, Sithamparanathan; Evans, Robin J.

doi:10.1109/lwc.2016.2600576

Cited by 13 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The neural network was able to rapidly learn the best user selection strategy in an unknown dynamic environment with a high success rate and fast convergence, as the findings also revealed. Arunthavanathan et al [11] demonstrated that a Markov decision process (MDP) might be used to make decisions based on the current transmissions in the channel. Decisions were made and the effects of interference and waste were evaluated for a range of occupancy rates.…”

Section: Survey On Learning-based Allocationmentioning

confidence: 99%

See 1 more Smart Citation

Elite-CAM: An Elite Channel Allocation and Mapping for Policy Engine over Cognitive Radio Technology in 5G

Babu

Balakrishnan

Ramana

et al. 2022

Sensors

View full text Add to dashboard Cite

The spectrum allocation in any auctioned wireless service primarily depends upon the necessity and the usage of licensed primary users (PUs) of a certain band of frequencies. These frequencies are utilized by the PUs as per their needs and requirements. When the allocated spectrum is not being utilized in the full efficient manner, the unused spectrum is treated by the PUs as white space without believing much in the concept of spectrum scarcity. There are techniques invented and incorporated by many researchers, such as cognitive radio technology, which involves software-defined radio with reconfigurable antennas tuned to particular frequencies at different times. Cognitive radio (CR) technology realizes the logic of the utility factor of the PUs and the requirements of the secondary users (SU) who are in queue to utilize the unused spectrum, which is the white space. The CR technology is enriched with different frequency allocation engines and with different strategies in different parts of the world, complying with the regulatory standards of the FCC and ITU. Based on the frequency allocation made globally, the existing CR technology understands the nuances of static and dynamic spectrum allocation and also embraces the intelligence in time allocation by scheduling the SUs whenever the PUs are not using the spectrum, and when the PUs pitch in the SUs have to leave the band without time. This paper identifies a few of the research gaps existing in the earlier literature. The behavioral aspects of the PUs and SUs have been analyzed for a period of 90 days with some specific spectrum ranges of usage in India. The communal habits of utilizing the spectrum, not utilizing the spectrum as white space, different time zones, the requisites of the SUs, the necessity of the applications, and the improvement of the utility factor of the entire spectrum have been considered along with static and dynamic spectrum usage, the development of the spectrum policy engine aligned with cooperative and opportunistic spectrum sensing, and access techniques indulging in artificial intelligence (AI). This will lead to fine-tuning the PU and SU channel mapping without being hindered by predefined policies. We identify the cognitive radio transmitter and receiver parameters, and resort to the same in a proposed channel adaption algorithm. We also analyze the white spaces offered by spectrum ranges of VHF, GSM-900, and GSM-1800 by a real-time survey with a spectrum analyzer. The identified parameters and white spaces are mapped with the help of a swotting algorithm. A sample policy has been stated for ISM band 2.4 GHz where such policies can be excited in a policy server. The policy engine is suggested to be configured over the 5G CORE spectrum management function.

show abstract

Section: Survey On Learning-based Allocationmentioning

confidence: 99%

“…The anticipated spectrum characteristic needs for secondary users (bandwidth Ba, power dB req, time access period, SNR) for various types of spectral intensity, geographic location, and regulatory authority [11] R…”

mentioning

confidence: 99%

Elite-CAM: An Elite Channel Allocation and Mapping for Policy Engine over Cognitive Radio Technology in 5G

Babu

Balakrishnan

Ramana

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…As demonstrated in [18], the interaction process between the agent and environment in reinforcement learning can be defined as a Markov decision process, as shown in Fig. 1.…”

Section: A Agent-environment Interfacementioning

confidence: 99%

An Improved Sarsa($\lambda$ ) Reinforcement Learning Algorithm for Wireless Communication Systems

et al. 2019

View full text Add to dashboard Cite

In this article, we provide a novel improved model-free temporal-difference control algorithm, namely, Expected Sarsa(λ), using the average value as an update target and introducing eligibility traces in wireless communication networks. In particular, we construct the update target using the average action value of all possible successive actions, and apply eligibility traces to record the historical access of every state action pair, which greatly improve the model's convergence property and learning efficiency. Numerical results demonstrate that the proposed algorithm has the advantage of high learning efficiency and a higher learning-rate tolerance range than Q Learning, Sarsa, Expected Sarsa, and Sarsa(λ) in the tabular case of a finite Markov decision process, thereby providing an efficient solution for the study and design wireless communication networks. This provides an efficient and effective solution to design further artificial intelligent communication networks. INDEX TERMS Model-free reinforcement learning, Sarsa, Q learning, eligibility traces. I. INTRODUCTION

show abstract

“…In a cognitive radio sensor network (CRSN), cognitive functionality is integrated with sensor nodes to form a new sensor networking paradigm. Therefore, allowing unlicensed sensor nodes to broadcast in the licensed band was proposed as a way to increase spectrum efficiency and accommodate users in the unlicensed spectrum band [ 6 ]. A cognitive radio user (CRU) detects one or more empty channels to avoid interference in the first stage prior to CR communication.…”

Section: Introductionmentioning

confidence: 99%

Multi-Radio Based Rendezvous Technique for Heterogeneous Cognitive Radio Sensor Network

Islam

Kandeepan

Evans

2021

Sensors

Self Cite

View full text Add to dashboard Cite

In a distributed cognitive radio (CR) sensor network, transmission and reception on vacant channels require cognitive radio nodes to achieve rendezvous. Because of the lack of adequate assistance from the network environment, such as the central controller and other nodes, assisted rendezvous for distributed CR is inefficient in a dynamic network. As a result, non-assisted blind rendezvous, which is unaware of its counterpart node, has recently led to a lot of interest in the research arena. In this paper, we study a channel rendezvous method based on prime number theory and propose a new multi-radio-based technique for non-assisted rendezvous with the blind and heterogeneous condition. The required time and the optimal number of radios for the guaranteed rendezvous are calculated using probability-based measurement. Analytical expressions for probabilistic guaranteed rendezvous conditions are derived and verified by Monte Carlo simulation. In addition, the maximum time to rendezvous (MTTR) is derived in closed form using statistical and probabilistic analysis. Under different channel conditions, our proposed solution leads to a substantial time reduction for guaranteed rendezvous. For the sake of over-performance of our proposed system, the simulation outcome is compared to a recently proposed heterogeneous and blind rendezvous method. The Matlab simulation results show that our proposed system’s MTTR gains range from 11% to over 95% for various parametric values of the system model.

show abstract

A Markov Decision Process-Based Opportunistic Spectral Access

Cited by 13 publications

References 11 publications

Elite-CAM: An Elite Channel Allocation and Mapping for Policy Engine over Cognitive Radio Technology in 5G

Elite-CAM: An Elite Channel Allocation and Mapping for Policy Engine over Cognitive Radio Technology in 5G

An Improved Sarsa($\lambda$ ) Reinforcement Learning Algorithm for Wireless Communication Systems

Multi-Radio Based Rendezvous Technique for Heterogeneous Cognitive Radio Sensor Network

Contact Info

Product

Resources

About