Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning

Guo, Delin; Tang, Lan; Zhang, Xinggan; Liang, Ying-Chang

doi:10.1109/tvt.2020.3020400

Cited by 119 publications

(45 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed model uses the received signal measurements to reduce the number of unnecessary HOs and predict the user location while ensuring that the throughput of the network is maintained. A joint optimization framework for minimizing HO frequency and maximizing user throughput was proposed in [178]. The HO and power allocation problem was modeled as a cooperative multi-agent task, after which a MARL framework using proximal policy optimization (PPO) was developed.…”

Section: ) Wireless Data Aided Handover Optimizationmentioning

confidence: 99%

A Survey of Machine Learning Applications to Handover Management in 5G and Beyond

et al. 2021

View full text Add to dashboard Cite

Handover (HO) is one of the key aspects of next-generation (NG) cellular communication networks that need to be properly managed since it poses multiple threats to quality-of-service (QoS) such as the reduction in the average throughput as well as service interruptions. With the introduction of new enablers for fifth-generation (5G) networks, such as millimetre wave (mm-wave) communications, network densification, Internet of things (IoT), etc., HO management is provisioned to be more challenging as the number of base stations (BSs) per unit area, and the number of connections has been dramatically rising. Considering the stringent requirements that have been newly released in the standards of 5G networks, the level of the challenge is multiplied. To this end, intelligent HO management schemes have been proposed and tested in the literature, paving the way for tackling these challenges more efficiently and effectively. In this survey, we aim at revealing the current status of cellular networks and discussing mobility and HO management in 5G alongside the general characteristics of 5G networks. We provide an extensive tutorial on HO management in 5G networks accompanied by a discussion on machine learning (ML) applications to HO management. A novel taxonomy in terms of the source of data to be utilized in training ML algorithms is produced, where two broad categories are considered; namely, visual data and network data. The stateof-the-art on ML-aided HO management in cellular networks under each category is extensively reviewed with the most recent studies, and the challenges, as well as future research directions, are detailed. INDEX TERMS Handover, machine learning, mobility management, fifth generation.

show abstract

Section: ) Wireless Data Aided Handover Optimizationmentioning

confidence: 99%

A Survey of Machine Learning Applications to Handover Management in 5G and Beyond

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A handover penalty is added to the reward function to reduce the redundant handovers. Similarly, in [15] the authors propose a proximal policy optimisation method based on a multi-agent reinforcement learning algorithm for handover control and power allocation in HetNets. The handover in [15] is triggered based on a hybrid state vector, i.e., the UE current serving cell, UE's signal measurement, and the number of UEs served by each BS.…”

Section: Intelligent Mobility Management Controlmentioning

confidence: 99%

“…Similarly, in [15] the authors propose a proximal policy optimisation method based on a multi-agent reinforcement learning algorithm for handover control and power allocation in HetNets. The handover in [15] is triggered based on a hybrid state vector, i.e., the UE current serving cell, UE's signal measurement, and the number of UEs served by each BS. Zhou et al in [16] propose two mobility management controls based on an online-learning algorithm, i.e., upper-confidence bound policies.…”

Section: Intelligent Mobility Management Controlmentioning

confidence: 99%

Autonomous Mobility Management for 5G Ultra-Dense HetNets via Reinforcement Learning With Tile Coding Function Approximation

et al. 2021

View full text Add to dashboard Cite

Mobility management is an important feature in modern wireless networks that can provide seamless and ubiquitous connectivity to mobile users. Due to the dense deployment of small cells and heterogeneous network topologies, the traditional handover control method can lead to various mobility-related problems, such as frequent handovers and handover failures. On the other hand, the mobility management's maintenance and operation cost is also increased due to increasing node density. In this paper, an autonomous mobility management control approach is proposed to increase the mobility robustness of user equipment (UE) mobility and minimize the operational cost of mobility management. The proposed method is based on reinforcement learning, which can autonomously learn an optimal handover control policy by interacting with the environment. The function approximation approach is adopted to allow reinforcement learning to process a large state and action space. A linear function approximator is used to approximate the state-action value function. Finally, the semi-gradient Sarsa method is implemented to update the approximated state-action function and learn the optimal handover control policy. The simulation results show that the proposed method can effectively improve the mobility robustness of UE under different speed ranges. Compared with the conventional reference signal received power (RSRP) based approach, the proposed approach can reduce unnecessary handovers by about 20% and latency by 58%, while achieving near zero handover failure rate, and increasing throughput by 12%.

show abstract

“…Motivated by the deep reinforcement learning (DRL) approach in solving the dynamic problem [13], DRL has been applied into cache policies to improve the cache performance of dynamic cache scenarios. In [14], a DRL approach was proposed to reduce the transmission cost by jointly considering proactive cache and content recommendations.…”

Section: Introductionmentioning

confidence: 99%

A Smart Cache Content Update Policy Based on Deep Reinforcement Learning

Kwong

Liu

et al. 2020

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

This paper proposes a DRL-based cache content update policy in the cache-enabled network to improve the cache hit ratio and reduce the average latency. In contrast to the existing policies, a more practical cache scenario is considered in this work, in which the content requests vary by both time and location. Considering the constraint of the limited cache capacity, the dynamic content update problem is modeled as a Markov decision process (MDP). Besides that, the deep Q-learning network (DQN) algorithm is utilised to solve the MDP problem. Specifically, the neural network is optimised to approximate the Q value where the training data are chosen from the experience replay memory. The DQN agent derives the optimal policy for the cache decision. Compared with the existing policies, the simulation results show that our proposed policy is 56%–64% improved in terms of the cache hit ratio and 56%–59% decreased in terms of the average latency.

show abstract

Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning

Cited by 119 publications

References 24 publications

A Survey of Machine Learning Applications to Handover Management in 5G and Beyond

A Survey of Machine Learning Applications to Handover Management in 5G and Beyond

Autonomous Mobility Management for 5G Ultra-Dense HetNets via Reinforcement Learning With Tile Coding Function Approximation

A Smart Cache Content Update Policy Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About