Cognitive spectrum management in dynamic cellular environments: A case-based Q-learning approach

Morozs, Nils; Clarke, Tim; Grace, David

doi:10.1016/j.engappai.2016.07.002

Cited by 14 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their scheme's convergence took place after 13,000-20,000 iterations at 28 m/s vehicle speed. Morozs et al [188] proposed a scheme that integrates distributed Q-learning and CBR to facilitate several learning processes running in parallel. The RL method was considered for the CR network with RF energy harvesting in [189].…”

Section: ) Spectrum Sensingmentioning

confidence: 99%

Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks

et al. 2020

View full text Add to dashboard Cite

Nowadays, machine learning (ML), which is one of the most rapidly growing technical tools, is extensively used to solve critical challenges in various domains. Vehicular ad hoc network (VANET) is expected to be the key role player in reducing road casualties and traffic congestion. To ensure this role, a gigantic amount of data should be exchanged. However, current allocated wireless access for VANET is inadequate to handle such massive data amounts. Therefore, VANET faces a spectrum scarcity issue. Cognitive radio (CR) is a promising solution to overcome such an issue. CR-based VANET or CR-VANET must achieve several performance enhancement measures, including ultra-reliable and lowlatency communication. ML methods can be integrated with CR-VANET to make CR-VANET highly intelligent, achieve rapid adaptability to the dynamicity of the environment, and improve the quality of service in an energy-efficient manner. This paper presents an overview of ML, CR, VANET, and CR-VANET, including their architectures, functions, challenges, and open issues. The applications and roles of ML methods in CR-VANET scenarios are reviewed. Insights into the use of ML for autonomous or driverless vehicles are also presented. Current advancements in the amalgamation of these prominent technologies and future research directions are discussed. INDEX TERMS Machine learning, VANET, cognitive radio, autonomous vehicles, smart transportation system. performance of CR-VANET [15]. Security enhancement is one of the major issues in CR-VANET. Here, a vehicle can pretend to be a PU and propagate false information to obtain spectrum access selfishly. ML can be used to detect such actions and enhance security [16], [17]. ML also provides an optimum route to CR-VANET users to avoid traffic jams and road accidents. ML can also play a vital role in the best infotainment experience in CR-VANET. It can be used for appropriate scheduling, selecting the best channel, and prioritizing messages. CR and ML can play a major role in the next-generation driverless car system. The role of CR in the next-generation transportation system has been presented in previous discussions. This survey shows how ML can be applied to reduce road accidents and traffic congestion. CR can be used to accommodate the spectrum required to support massive data communication among automated driverless vehicles and networks. ML can be an integral part of this driverless or automated vehicle system. Similar to a robot, an autonomous vehicle (AV) can learn the surrounding environment and communicate with increased safety, reliability, QoS, and energy efficiency by applying such learning. This paper presents the dynamic usages of ML in CR-VANET elaborately. Several of the benefits of CR in VANETs and ML in VANETs and CR-VANETs are presented in Table 1. B. CONTRIBUTIONS OF THIS SURVEY ARTICLE Many survey articles describe CR, VANET, ML, and CR-VANETs individually or describe a few aspects of their amalgamation. To the best of our knowledge, surveys that cover the usage of ML in CR-VANET...

show abstract

Section: ) Spectrum Sensingmentioning

confidence: 99%

Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…A critical duty of cellular systems is to efficiently manage spectrum to maintain an acceptable QoS level among users in particularly voice calls and data transmissions. Due to inherent dynamic nature of such a cellular system, an intelligent RL-based approach is proposed to develop a dynamic spectrum access improving the performance of cognitive cellular systems [28]. Another application area improved with RL-based techniques is the cognitive radio.…”

Section: Rl-based Applicationsmentioning

confidence: 99%

SARL: A reinforcement learning based QoS-aware IoT service discovery model

Koşunalp

Demir

2020

Journal of Electrical Engineering

View full text Add to dashboard Cite

The IoT environment includes the enormous amount of atomic services with dynamic QoS compared with traditional web services. In such an environment, in the service composition process, discovering a requested service meeting the required QoS is a di cult task. In this work, to address this issue, we propose a peer-to-peer-based service discovery model, which looks for the information about services meeting the requested QoS and functionality on an overlay constructed with users of services versus service nodes, with probably constrained resources. However, employing a plain discovery algorithm on the overlay network such as flooding, or k-random walk could cause high message overhead or delay. This necessitates an intelligent and adaptive discovery algorithm, which adapts itself based on users’ previous queries and the results. To fill this gap, the proposed service discovery approach is equipped with a reinforcement learning-based algorithm, named SARL. The reinforcement learning-based algorithm enables SARL to significantly reduce delay and message overhead in the service discovery process by ranking neighboring nodes based on users’ service request preferences and the service query results. The proposed model is implemented on the OMNet simulation platform. The simulation results demonstrate that SARL remarkably outperforms the existing approaches in terms of message overhead, reliability, timeliness, and energy usage efficiency.

show abstract

“…Reinforcement learning (RL) technique has been implemented in various contexts such as dynamic spectrum access in cellular networks with dynamically changing topologies [20], management of inter‐cell interference coordination among neighbouring BSs in downlink of Long‐Term Evolution network [21], channel access for WiFi system at the unlicensed band for coexistence with the Long‐Term Evolution‐time division duplex system [22], cognitive sensor network [23], power allocation in the cognitive wireless mess network [24], interference mitigation in femto‐cell deployment [25] etc. However, none of the works has implemented the RL technique in the context of negotiation in spectrum trading involving multiple issues.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement learning‐based negotiation for spectrum micro‐trading framework

2018

View full text Add to dashboard Cite

Spectrum trading has been permitted in most of the major wireless markets to facilitate better utilisation of spectrum. The authors have considered a spectrum trading framework in which a wireless service provider (WSP) leases its available channel(s) to another WSP for use at the designated base station (BS) of the latter for short duration. In their model, the agents of WSPs carry out the negotiation on the specifications of the channel usage such as transmission power, antenna height, spectrum band of the channel and price of the channel. In this work, they have modelled the negotiation as a multi-issue bilateral negotiation problem. Initially, they have solved the problem with the help of the Bayesian learning-based negotiation (BLBN) method. Furthermore, they have devised the novel reinforcement learning-based technique namely, reinforcement learning-based negotiation (RLBN) considering the adaptability of the BS to the new channel configuration. Surplus utility and convergence time of the negotiation process are considered as performance indices for the above techniques. The simulation results show that the RLBN outperforms BLBN and static negotiation technique as far as the objective of surplus utility is concerned.

show abstract

Cognitive spectrum management in dynamic cellular environments: A case-based Q-learning approach

Cited by 14 publications

References 29 publications

Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks

Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks

SARL: A reinforcement learning based QoS-aware IoT service discovery model

Reinforcement learning‐based negotiation for spectrum micro‐trading framework

Contact Info

Product

Resources

About