A Model-Driven Deep Reinforcement Learning Heuristic Algorithm for Resource Allocation in Ultra-Dense Cellular Networks

Liao, Xiaomin; Shi, Jia; Li, Zan; Zhang, Lei; Xia, Baiqiang

doi:10.1109/tvt.2019.2954538

Cited by 52 publications

(20 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In another interesting work in [162], the authors propose a model-driven multi-agent Double DQN-based framework for resource allocation in UDNs. In particular, They first develop a DNN-based optimization framework comprised of a series of ADMM iterative procedures that uses the CSI as the learned weights.…”

Section: ) In Cellular and Homnetsmentioning

confidence: 99%

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Alwarafy¹,

Abdallah²,

Çiftler³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div>Next generation wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they incorporate, the types and numbers of smart IoT devices they serve, and the types of emerging applications they support. In such large-scale and heterogeneous networks (HetNets), radio resource allocation and management (RRAM) becomes one of the major challenges encountered during system design and deployment. In this context, emerging Deep Reinforcement Learning (DRL) techniques are expected to be one of the main enabling technologies to address the RRAM in future wireless HetNets. In this paper, we conduct a systematic in-depth, and comprehensive survey of the applications of DRL techniques in RRAM for next generation wireless networks. Towards this, we first overview the existing traditional RRAM methods and identify their limitations that motivate the use of DRL techniques in RRAM. Then, we provide a comprehensive review of the most widely used DRL algorithms to address RRAM problems, including the value- and policy-based algorithms. The advantages, limitations, and use-cases for each algorithm are provided. We then conduct a comprehensive and in-depth literature review and classify existing related works based on both the radio resources they are addressing and the type of wireless networks they are investigating. To this end, we carefully identify the types of DRL algorithms utilized in each related work, the elements of these algorithms, and the main findings of each related work. Finally, we highlight important open challenges and provide insights into several future research directions in the context of DRL-based RRAM. This survey is intentionally designed to guide and stimulate more research endeavors towards building efficient and fine-grained DRL-based RRAM schemes for future wireless networks.</div>

show abstract

Section: ) In Cellular and Homnetsmentioning

confidence: 99%

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Alwarafy¹,

Abdallah²,

Çiftler³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Compared with benchmark routing strategy, the proposed system outperforms in terms of signaling overhead, throughput, and delay. In [20], a DRL assisted resource allocation method is designed for ultra dense networks. The original multi-objective problem is decoupled into two parts based on the general theory of DRL.…”

Section: A Related Workmentioning

confidence: 99%

“…Based on the concept of opportunistically minimizing an expectation, the policy that minimizes E {P (t)|Q(t)} is the one that minimizes P (t) with the observation of Q(t) during each slot. Since neither Q i (t)A i (t) nor B in (20) will be affected by the policy at slot t, the upper bound minimization for the drift-plus-penalty can be accomplished by solving the following deterministic problem at slot t:…”

Section: B General Lyapunov Optimizationmentioning

confidence: 99%

Mode Selection and Resource Allocation in Sliced Fog Radio Access Networks: A Reinforcement Learning Approach

Xiang

Peng

Sun

et al. 2020

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

The mode selection and resource allocation in fog radio access networks (F-RANs) have been advocated as key techniques to improve spectral and energy efficiency. In this paper, we investigate the joint optimization of mode selection and resource allocation in uplink F-RANs, where both of the traditional user equipments (UEs) and fog UEs are served by constructed network slice instances. The concerned optimization is formulated as a mixed-integer programming problem, and both the orthogonal and multiplexed subchannel allocation strategies are proposed to guarantee the slice isolation. Motivated by the development of machine learning, two reinforcement learning based algorithms are developed to solve the original high complexity problem under traditional and fog UEs' specific performance requirements. The basic idea of the proposals is to generate a good mode selection policy according to the immediate reward fed back by an environment. Simulation results validate the benefits of our proposed algorithms and show that a tradeoff between system power consumption and queue delay can be achieved.Index Terms-fog radio access network, network slicing, reinforcement learning.

show abstract

“…is the value of r p k in the m th iteration of solving (23). We successively solve (23) with Algorithm 2 and update r m until r m converges.…”

Section: )mentioning

confidence: 99%

Resource Reservation in Backhaul and Radio Access Network with Uncertain User Demands

Reyhanian

Farmanbar

Luo

2020

2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)

View full text Add to dashboard Cite

Resource reservation is an essential step to enable wireless data networks to support a wide range of user demands. In this paper, we consider the problem of joint resource reservation in the backhaul and Radio Access Network (RAN) based on the statistics of user demands and channel states, and also network availability. The goal is to maximize the sum of expected traffic flow rates, subject to link and access point budget constraints, while minimizing the expected outage of downlinks. The formulated problem turns out to be non-convex and difficult to solve to global optimality. We propose an efficient Block Coordinate Descent (BCD) algorithm to approximately solve the problem. The proposed BCD algorithm optimizes the link capacity reservation in the backhaul using a novel multipath routing algorithm that decomposes the problem down to link-level and parallelizes the computation across backhaul links, while the reservation of transmission resources in RAN is carried out via a novel scalable and distributed algorithm based on Block Successive Upper-bound Minimization (BSUM). We prove that the proposed BCD algorithm converges to a Karush-Kuhn-Tucker (KKT) solution. Simulation results verify the efficiency and the efficacy of our BCD approach against two heuristic algorithms.

show abstract

A Model-Driven Deep Reinforcement Learning Heuristic Algorithm for Resource Allocation in Ultra-Dense Cellular Networks

Cited by 52 publications

References 39 publications

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Mode Selection and Resource Allocation in Sliced Fog Radio Access Networks: A Reinforcement Learning Approach

Resource Reservation in Backhaul and Radio Access Network with Uncertain User Demands

Contact Info

Product

Resources

About