Deep Multi-Agent Reinforcement Learning for Resource Allocation in D2D Communication Underlaying Cellular Networks

Zhang, Xu; Lin, Zuan; Ding, Beichen; Gu, Bo; Han, Yu

doi:10.23919/apnoms50412.2020.9237060

Cited by 11 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RRAM in such networks is one major concern, especially for mmWave-based cellular networks, as the D2D links require frequent link re-establishment to combat the high blockage rate. The authors in [172] propose a multi-agent Double DQNbased scheme to address the problem of joint subcarrier assignment and power allocation in D2D underlying 5G cellular networks. The agents in their model are the D2D pairs, whose action space is discrete, corresponding to determining the transmit power allocation on the available subcarriers.…”

Section: ) In Cellular Networkmentioning

confidence: 99%

“…In particular, depending on the type of radio resources under investigation, resources with continuous nature such as power typically implement policy-based algorithms, while resources with discreet nature such as channel allocation and user association typically implement value-based algorithms. Simultaneous dealing with continuous and discrete types of radio resources may integrate both the policy-and value-based DRL algorithms to learn a global policy as in [41], [182], or even adopting the value-based algorithms as in, e.g., [172], [174], [185] with an expense of added quantization error.…”

Section: Findings and Lessons Learnedmentioning

confidence: 99%

See 1 more Smart Citation

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

Alwarafy

Abdallah

Çiftler

et al. 2022

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Next generation wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they incorporate, the types and numbers of smart IoT devices they serve, and the types of emerging applications they support. In such large-scale and heterogeneous networks (HetNets), radio resource allocation and management (RRAM) becomes one of the major challenges encountered during system design and deployment. In this context, emerging Deep Reinforcement Learning (DRL) techniques are expected to be one of the main enabling technologies to address the RRAM in future wireless HetNets. In this paper, we conduct a systematic in-depth, and comprehensive survey of the applications of DRL techniques in RRAM for next generation wireless networks. Towards this, we first overview the existing traditional RRAM methods and identify their limitations that motivate the use of DRL techniques in RRAM. Then, we provide a comprehensive review of the most widely used DRL algorithms to address RRAM problems, including the value-and policy-based algorithms. The advantages, limitations, and use-cases for each algorithm are provided. We then conduct a comprehensive and in-depth literature review and classify existing related works based on both the radio resources they are addressing and the type of wireless networks they are investigating. To this end, we carefully identify the types of DRL algorithms utilized in each related work, the elements of these algorithms, and the main findings of each related work. Finally, we highlight important open challenges and provide insights into several future research directions in the context of DRL-based RRAM. This survey is intentionally designed to guide and stimulate more research endeavors towards building efficient and fine-grained DRL-based RRAM schemes for future wireless networks.

show abstract

Section: ) In Cellular Networkmentioning

confidence: 99%

Section: Findings and Lessons Learnedmentioning

confidence: 99%

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

Alwarafy

Abdallah

Çiftler

et al. 2022

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

show abstract

“…The authors in [164] propose a multi-agent Double DQNbased scheme to address the problem of joint subcarrier assignment and power allocation in D2D underlying 5G cellular networks. The agents in their model are the D2D pairs, whose action space is discrete, corresponding to determining the transmit power allocation on the available subcarriers.…”

Section: ) In Cellular and Homnetsmentioning

confidence: 99%

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Alwarafy¹,

Abdallah²,

Çiftler³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div>Next generation wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they incorporate, the types and numbers of smart IoT devices they serve, and the types of emerging applications they support. In such large-scale and heterogeneous networks (HetNets), radio resource allocation and management (RRAM) becomes one of the major challenges encountered during system design and deployment. In this context, emerging Deep Reinforcement Learning (DRL) techniques are expected to be one of the main enabling technologies to address the RRAM in future wireless HetNets. In this paper, we conduct a systematic in-depth, and comprehensive survey of the applications of DRL techniques in RRAM for next generation wireless networks. Towards this, we first overview the existing traditional RRAM methods and identify their limitations that motivate the use of DRL techniques in RRAM. Then, we provide a comprehensive review of the most widely used DRL algorithms to address RRAM problems, including the value- and policy-based algorithms. The advantages, limitations, and use-cases for each algorithm are provided. We then conduct a comprehensive and in-depth literature review and classify existing related works based on both the radio resources they are addressing and the type of wireless networks they are investigating. To this end, we carefully identify the types of DRL algorithms utilized in each related work, the elements of these algorithms, and the main findings of each related work. Finally, we highlight important open challenges and provide insights into several future research directions in the context of DRL-based RRAM. This survey is intentionally designed to guide and stimulate more research endeavors towards building efficient and fine-grained DRL-based RRAM schemes for future wireless networks.</div>

show abstract

“…The idea of leveraging the neighbors' information for centralized training is also considered in [81]. However, different from [80] that uses the BSs to collect information, the authors in [81] consider that each D2D user is able to collect information from neighbors, including interfering neighbors' states and interfered neighbors' states. These information can then be fed to a DDQN based agent model to learn an optimal spectrum selection policy.…”

Section: A Dynamic Spectrum Accessmentioning

confidence: 99%

“…The simulation results demonstrate that the proposed dedicated DQN is able to achieve better results than that of the shared DQN used in [89] and the random baseline in terms of sum capacity and the successful transmission probability of the V2V links. The agent information collecting mechanism utilized in the above works for centralized training, e.g., [80] and [81], do not consider a preprocess of state information, which may lead to extra signaling overhead and increase the computational complexity at the BS. The state information preprocess is investigated in [58].…”

Section: A Dynamic Spectrum Accessmentioning

confidence: 99%

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Li¹,

Zhu²,

Luong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in the emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues. Finally, we discuss the challenges, open issues, and future directions related to the applications of MARL in future Internet.

show abstract

Deep Multi-Agent Reinforcement Learning for Resource Allocation in D2D Communication Underlaying Cellular Networks

Cited by 11 publications

References 10 publications

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Contact Info

Product

Resources

About