Learning Radio Resource Management in RANs: Framework, Opportunities, and Challenges

Calabrese, Francesco; Wang, Li; Ghadimi, Euhanna; Peters, Gunnar; Hanzo, Lajos; Soldati, Pablo

doi:10.1109/mcom.2018.1701031

Cited by 144 publications

(109 citation statements)

References 11 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…The issue is then how to avoid catastrophic actions while the agents are actively exploring the environment and no concrete policies have been obtained yet. A seemingly good answer might be to use expert human knowledge to help confine the exploration space [83] and guide the learning agent's search within the space. But exactly how to implement the concept in algorithm design with performance guarantee is unclear and worth further investigation.…”

Section: B Bridging the Gap Between Training And Implementationmentioning

confidence: 99%

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

et al. 2020

View full text Add to dashboard Cite

It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, e.g., in the highmobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this paper, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep learning assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.

show abstract

Section: B Bridging the Gap Between Training And Implementationmentioning

confidence: 99%

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…RL is eminently suitable for solving problems formulated as Markov decision processes (MDPs), e.g., distributed resource optimization [10], rather than the variable optimization problems formulated as P1 of Fig. 1.…”

Section: B Functional Optimization Using Unsupervised Learning and Rmentioning

confidence: 99%

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

Resource allocation and transceivers in wireless networks are usually designed by solving optimization problems subject to specific constraints, which can be formulated as variable or functional optimization. If the objective and constraint functions of a variable optimization problem can be derived, standard numerical algorithms can be applied for finding the optimal solution, which however incur high computational cost when the dimension of the variable is high. To reduce the online computational complexity, learning the optimal solution as a function of the environment's status by deep neural networks (DNNs) is an effective approach. DNNs can be trained under the supervision of optimal solutions, which however, is not applicable to the scenarios without models or for functional optimization where the optimal solutions are hard to obtain. If the objective and constraint functions are unavailable, reinforcement learning can be applied to find the solution of a functional optimization problem, which is however not tailored to optimization problems in wireless networks. In this article, we introduce unsupervised and reinforced-unsupervised learning frameworks for solving both variable and functional optimization problems without the supervision of the optimal solutions. When the mathematical model of the environment is completely known and the distribution of environment's status is known or unknown, we can invoke unsupervised learning algorithm. When the mathematical model of the environment is incomplete, we introduce reinforcedunsupervised learning algorithms that learn the model by interacting with the environment. Our simulation results confirm the applicability of these learning frameworks by taking a user association problem as an example.

show abstract

“…In (6), only a single center agent is trained and then implemented. Under this framework, the current local channel state information (CSI) is first estimated and transmitted to the center agent In [36], a framework of centralized training and distributed execution was proposed to address these challenges. The power allocation scheme is decentralized, the transmitter of each link is regarded as an agent, and all agents in the communication network operate synchronously and distributively.…”

Section: B Centralized Training and Distributed Executionmentioning

confidence: 99%

Power Allocation in Multi-User Cellular Networks with Deep Q Learning Approach

Meng

Chen

2019

ICC 2019 - 2019 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

The model-based power allocation algorithm has been investigated for decades, but it requires the mathematical models to be analytically tractable and it usually has high computational complexity.Recently, the data-driven model-free machine learning enabled approaches are being rapidly developed to obtain near-optimal performance with affordable computational complexity, and deep reinforcement learning (DRL) is regarded as of great potential for future intelligent networks. In this paper, the DRL approaches are considered for power control in multi-user wireless communication cellular networks.Considering the cross-cell cooperation, the off-line/on-line centralized training and the distributed execution, we present a mathematical analysis for the DRL-based top-level design. The concrete DRL design is further developed based on this foundation, and policy-based REINFORCE, value-based deep Q learning (DQL), actor-critic deep deterministic policy gradient (DDPG) algorithms are proposed.Simulation results show that the proposed data-driven approaches outperform the state-of-art modelbased methods on sum-rate performance, with good generalization power and faster processing speed.Furthermore, the proposed DDPG outperforms the REINFORCE and DQL in terms of both sum-rate performance and robustness, and can be incorporated into existing resource allocation schemes due to its generality.Deep reinforcement learning, deep deterministic policy gradient, policy-based, interfering multipleaccess channel, power control, resource allocation. I. INTRODUCTIONWireless data transmission has experienced tremendous growth in past years and will continue to grow in the future. When large numbers of terminals such as mobile phones and wearable devices are connected to the networks, the density of access point (AP) will have to be increased. Dense deployment of small cells such as pico-cells, femto-cells, has become the most effective solution to accommodate the critical demand for spectrum [1]. With denser APs and smaller cells, the whole communication network is flooded with wireless signals, and thus the intra-cell and inter-cell interference problems are severe [2]. Therefore, power allocation and interference management are crucial and challenging [3], [4].Massive model-oriented algorithms have been developed to cope with interference management [5]- [9], and the existing studies mainly focus on sub-optimal or heuristic algorithms, whose performance gaps to the optimal solution are typically difficult to quantify. Besides, the mathematical models are usually assumed to be analytically tractable, but these models are not always accurate because both hardware and channel imperfections can exist in practical communication environments. When considering specific hardware components and realistic transmission scenarios, such as low-resolution A/D, nonlinear amplifier and user distribution, the signal processing techniques with model-driven tools are challenging to be developed. Moreover, the computational complexity of these algorithms i...

show abstract

Learning Radio Resource Management in RANs: Framework, Opportunities, and Challenges

Cited by 144 publications

References 11 publications

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

Power Allocation in Multi-User Cellular Networks with Deep Q Learning Approach

Contact Info

Product

Resources

About