RAN Resource Slicing in 5G Using Multi-Agent Correlated Q-Learning

Abstract

5G is regarded as a revolutionary mobile network, which is expected to satisfy a vast number of novel services, ranging from remote health care to smart cities. However, heterogeneous Quality of Service (QoS) requirements of different services and limited spectrum make the radio resource allocation a challenging problem in 5G. In this paper, we propose a multi-agent reinforcement learning (MARL) method for radio resource slicing in 5G. We model each slice as an intelligent agent that competes for limited radio resources, and the correlated Q-learning is applied for inter-slice resource block (RB) allocation. The proposed correlated Q-learning based interslice RB allocation (COQRA) scheme is compared with Nash Q-learning (NQL), Latency-Reliability-Throughput Q-learning (LRTQ) methods, and the priority proportional fairness (PPF) algorithm. Our simulation results show that the proposed CO-QRA achieves 32.4% lower latency and 6.3% higher throughput when compared with LRTQ, and 5.8% lower latency and 5.9% higher throughput than NQL. Significantly higher throughput and lower packet drop rate (PDR) is observed in comparison to PPF.

Team Learning-Based Resource Allocation for Open Radio Access Network (O-RAN)

“…For example, in [8], a DQN-based algorithms is proposed for power allocation in wireless networks. [9] proposed an correlated Q-learning algorithm to optimally allocate resources for network slicing. In [10], the authors designed an algorithm to learn the optimal handover control strategy by using deep neural networks (DNNs).…”

Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

“…The aforementioned works show that various ML methods have been applied for the resource management of wireless networks, including RL [12], [13], [16], [17], DRL [14], double deep Q-learning [15], multi-agent RL [18], multiagent DRL [19], actor-critic DRL [20], and so on. In our former work [21], we proposed a correlated Q-learning based method for the radio resource allocation of 5G RAN, but the knowledge transfer was still not considered. In these works, the main motivations of deploying ML algorithms are the increasing complexity of wireless networks and the difficulties to build dedicated optimization models.…”

Knowledge Transfer based Radio and Computation Resource Allocation for 5G RAN Slicing

“…Deep reinforcement learning is considered as a breakthrough, but the time-consuming network training is a well known issue [17]. We propose a correlated Q-learning based method for radio resource allocation of network slicing in [18], but the knowledge transfer capability is still not considered. To this end, we propose a KTRA scheme for the joint radio and computation resources allocation of 5G networks in this work.…”

“…The KTRA algorithm is summarized as Algorithm 1. Noting that we apply a classic proportional fairness algorithm for intra-slice radio resource allocation, because this work mainly focus on inter-slice level scheduling [18], and there is no intraslice computation resource allocation.…”