Deep Reinforcement Learning-Based Dynamic Spectrum Access for D2D Communication Underlay Cellular Networks

Huang, Jingfei; Yang, Yang; He, Gang; Xiao, Yang; Li, Jun

doi:10.1109/lcomm.2021.3079920

Cited by 24 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Introductionmentioning

confidence: 99%

“…Many research works have applied learning techniques to RA for D2D communications [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ]. As a learning principle for training RA units, DL [ 18 , 19 , 20 , 21 , 22 ], RL [ 23 , 24 , 25 ], and DRL [ 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ] have been widely utilized. Depending on who determines the resource allocations for D2D devices, two types of RA schemes have been proposed: a centralized RA [ 18 , 19 , 20 , 23 , 26 , 27 , 28 , 31 ] and a decentralized RA [ 20 , 21 , 22 , 23 , 25 , 29 , 30 , 32 , 33 , 34 ].…”

Section: Introductionmentioning

confidence: 99%

“…As a learning principle for training RA units, DL [ 18 , 19 , 20 , 21 , 22 ], RL [ 23 , 24 , 25 ], and DRL [ 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ] have been widely utilized. Depending on who determines the resource allocations for D2D devices, two types of RA schemes have been proposed: a centralized RA [ 18 , 19 , 20 , 23 , 26 , 27 , 28 , 31 ] and a decentralized RA [ 20 , 21 , 22 , 23 , 25 , 29 , 30 , 32 , 33 , 34 ]. In the case of DRL-based RA schemes, a single-agent framework is used for centralized RA schemes [ 26 , 27 , 28 , 31 ] while a multi-agent framework is used for decentralized RA schemes [ 21 , 22 , 23 , 24 , 25 , 29 , 30 , 32 , 33 , 34 ].…”

Section: Introductionmentioning

confidence: 99%

“…Depending on who determines the resource allocations for D2D devices, two types of RA schemes have been proposed: a centralized RA [ 18 , 19 , 20 , 23 , 26 , 27 , 28 , 31 ] and a decentralized RA [ 20 , 21 , 22 , 23 , 25 , 29 , 30 , 32 , 33 , 34 ]. In the case of DRL-based RA schemes, a single-agent framework is used for centralized RA schemes [ 26 , 27 , 28 , 31 ] while a multi-agent framework is used for decentralized RA schemes [ 21 , 22 , 23 , 24 , 25 , 29 , 30 , 32 , 33 , 34 ]. In essence, centralized single-agent RA schemes have attained a high QoS in the communication network by utilizing highly computational complexities.…”

Section: Introductionmentioning

confidence: 99%

“…An increasing number of research studies are investigating and devising RA schemes based on the DRL principle. A centralized double-DQN-based RA scheme was proposed for dynamic spectrum access in D2D communications underlay cellular networks [ 26 ], and a centralized hierarchical DRL-based method was proposed to find an optimal relay selection and power allocation strategy for 5G mmWave D2D links [ 27 ]. In [ 28 ], a DRL-based algorithm was proposed to determine the transmit power of D2D and cellular links for maximizing an overall sum-rate.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Deep Reinforcement Learning Based Resource Allocation for D2D Communications Underlay Cellular Networks

Lee

2022

Sensors

View full text Add to dashboard Cite

In this paper, a resource allocation (RA) scheme based on deep reinforcement learning (DRL) is designed for device-to-device (D2D) communications underlay cellular networks. The goal of RA is to determine the transmission power and spectrum channel of D2D links to maximize the sum of the average effective throughput of all cellular and D2D links in a cell accumulated over multiple time steps, where a cellular channel can be allocated to multiple D2D links. Allowing a cellular channel to be shared by multiple D2D links and considering performance over multiple time steps require a high level of system overhead and computational complexity so that optimal RA is practically infeasible in this scenario, especially when a large number of D2D links are involved. To mitigate the complexity, we propose a sub-optimal RA scheme based on a multi-agent DRL, which operates with shared information in participating devices, such as locations and allocated resources. Each agent corresponds to each D2D link and multiple agents perform learning in a staggered and cyclic manner. The proposed DRL-based RA scheme allocates resources to D2D devices promptly according to dynamically varying network set-ups, including device locations. The proposed sub-optimal RA scheme outperforms other schemes, where the performance gain becomes significant when the densities of devices in a cell are high.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep Reinforcement Learning Based Resource Allocation for D2D Communications Underlay Cellular Networks

Lee

2022

Sensors

View full text Add to dashboard Cite

show abstract

Deep reinforcement learning for resource allocation of mobile communication systems with device‐to‐device underlay

Cardoso

Carvalho

Gondim

2023

Int J Communication

View full text Add to dashboard Cite

SummaryOver the past few decades, the number of users and services of the mobile communications system has considerably increased, and since its essential resources such as spectrum and energy are limited, their optimization has drawn particular interest. Concomitantly, artificial intelligence (AI) techniques have advanced and their applications have been expanded, including problems of classification, regression, and optimization of tasks of mobile communications systems. Regarding fifth and sixth generations of such systems, the insertion of AI is foreseen toward the allocation of available resources. The present study applied two recently proposed techniques based on deep reinforcement learning algorithms (viz., deep deterministic policy gradient [DDPG] and twin‐delayed DDPG [TD3]), for the power control and spectrum allocation of a mobile communications system with device‐to‐device (D2D) underlay communications. The results show that both algorithms have superior performance to the three algorithms used for comparison: A random algorithm, a greedy algorithm, and REINFORCE, a classical reinforcement learning algorithm. Furthermore, the results show the proposed algorithms have good generalization capability and performed the allocation intelligently, taking into account the relationship between distances separating devices and interference between communications. The results also proved robust in terms of small variations in input data and noise.

show abstract

Distributed Dynamic Spectrum Access for D2D Communications Underlying Cellular Networks Using Deep Reinforcement Learning

Jiang¹,

Han²,

Wang³

2023

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Deep Reinforcement Learning-Based Dynamic Spectrum Access for D2D Communication Underlay Cellular Networks

Cited by 24 publications

References 9 publications

Deep Reinforcement Learning Based Resource Allocation for D2D Communications Underlay Cellular Networks

Deep Reinforcement Learning Based Resource Allocation for D2D Communications Underlay Cellular Networks

Deep reinforcement learning for resource allocation of mobile communication systems with device‐to‐device underlay

Distributed Dynamic Spectrum Access for D2D Communications Underlying Cellular Networks Using Deep Reinforcement Learning

Contact Info

Product

Resources

About