DDPG-Based Throughput Optimization with AoI Constraint in Ambient Backscatter-Assisted Overlay CRN

Abstract

The combination of ambient backscatter (AB) communications (ABCs) and RF-powered cognitive radio networks (CRNs) deals with challenges of both energy supply and spectrum shortage, and improves the network performances. With the expansion of wireless networks, many applications raise requirements for both high-throughput and timely data. Driven by these facts, we study the long-term throughput optimization of the secondary network in the AB-assisted overlay CRN (ABO-CRN), ABCs, and CRNs with the age of information (AoI) constraint, which is a novel metric for measuring the freshness of data received by receivers. Due to the dynamic environment, complete knowledge of the environment could not be obtained. Then, the deep deterministic policy gradient (DDPG), a deep reinforcement learning (DRL) method that addresses decision issues in both continuous and discrete spaces, is deployed to address the throughput optimization. We consider the impacts of time and energy allocation on the reward when the AoI constraint can not be satisfied, and develop the corresponding reward functions. Furthermore, we analyze the impacts of the minimum throughput requirement and maximum allowable AoI on the throughput and AoI of the secondary networks in the ABO-CRN, ABCs, and CRNs. We compare the throughput optimization scheme under the AoI constraint with two baseline schemes (i.e., throughput-optimal (T-O) and AoI-optimal (A-O) baseline schemes), and the simulation results show that the throughput of the ABO-CRN is close to the optimal throughput of the T-O baseline scheme, and the AoI of the ABO-CRN is close to the optimal AoI of the A-O baseline scheme.

AoI-Oriented Resource Allocation for NOMA-Based Wireless Powered Cognitive Radio Networks Based on Multi-Agent Deep Reinforcement Learning

“…For a WPCRN, the average AoI minimization problem was formulated as partially observable markov decision process and solved by dynamic programming in [44]. In [45], the network throughput maximization problem with AoI constraint was studied for a WPCRN with backscatter communication, and the DDPG algorithm was proposed. To the best of our knowledge, few research works have investigated the AoI-oriented resource allocation for NOMA-based WPCRN.…”