A POMDP‐based long‐term transmission rate maximization for cognitive radio networks with wireless‐powered ambient backscatter

Abstract

SummaryWireless energy harvesting enables wireless‐powered communications to accommodate data services in a self‐sustainable manner over a long operational time. Along with energy harvesting, an ambient backscatter technique helps a secondary transmitter reflect existing radio frequency (RF) signal sources to communicate with a secondary receiver when the primary channel (PC) is utilized. However, secondary system performance is significantly affected by factors such as the availability of the primary channel, imperfect spectrum sensing, and energy‐constrained problems. Therefore, we propose a novel approach for wireless‐powered cognitive radio networks (CRNs) to improve the transmission performance of secondary systems. To reduce the dependence of the secondary system on RF sources, in the paper, we provide a new paradigm by integrating ambient backscattering with both RF and non‐RF wireless‐powered communications to facilitate secondary communications. On the basis of the sensing result in a time slot, the secondary transmitter can dynamically select the operational action: (a) backscattering, (b) harvesting, or (c) transmitting to maximize the long‐term achievable data transmission rate at the secondary receiver. In addition, the optimal action set for CRNs with wireless‐powered ambient backscatter is selected by the partially observable Markov decision process (POMDP), which maximizes an expected transmission rate calculated over a number of subsequent time slots. The proposed scheme aims to improve long‐term transmission rate of CRNs with wireless‐powered ambient backscatter in comparison with conventional schemes where an action is taken only to maximize the immediate reward in every single time slot.

Hybrid NOMA/OMA-Based Dynamic Power Allocation Scheme Using Deep Reinforcement Learning in 5G Networks

“…Along with the rapid proliferation of wireless communication applications, most battery-limited devices become useless if their battery power is depleted. As one of the remedies, energy harvesting (EH) exploits ambient energy resources to replenish batteries, such as solar energy [20], radio frequency (RF) signals [21], and both non-RF and RF energy harvesting [22], etc. Among various kinds of renewable energy resources, solar power has been considered one of the most effective energy sources for wireless devices.…”

“…Since the information regarding network dynamics (e.g., harvested energy distribution, primary user's behavior) is sometimes unavailable in the cognitive radio system, researchers usually formulate optimization problems as the framework of a Markov decision process (MDP) [20,22,26,27]. Reinforcement learning is one of the potential approaches to obtaining the optimal solution for an MDP problem by interacting with the environment without having prior information about the network dynamics or without any supervision [28][29][30].…”