Flexible Resource Block Allocation to Multiple Slices for Radio Access Network Slicing Using Deep Reinforcement Learning

Abstract

In the fifth-generation of mobile communications, network slicing is used to provide an optimal network for various services as a slice. In this paper, we propose a radio access network (RAN) slicing method that flexibly allocates RAN resources using deep reinforcement learning (DRL). In RANs, the number of slices controlled by a base station fluctuates in terms of user ingress and egress from the base station coverage area and service switching on the respective sets of user equipment. Therefore, when resource allocation depends on the number of slices, resources cannot be allocated when the number of slices changes. We consider a method that makes optimal-resource allocation independent of the number of slices. Resource allocation is optimized using DRL, which learns the best action for a state through trial and error. To achieve independence from the number of slices, we show a design for a model that manages resources on a one-slice-by-one-agent basis using ApeX , which is a DRL method. In ApeX , because agents can be employed in parallel, models that learn various environments can be generated through trial and error of multiple environments. In addition, we design a model that satisfies the slicing requirements without overallocating resources. Based on this design, it is possible to optimally allocate resources independently of the number of slices by changing the number of agents. In the evaluation, we test multiple scenarios and show that the mean satisfaction of the slice requirements is approximately 97%. INDEX TERMS Deep reinforcement learning, network slicing, RAN slicing, resource management.

Intelligent multi-branch allocation of spectrum slices for inter-numerology interference minimization

“…However, although [14] and [15] consider a mixed-numerology access scheme, the INI impact on the algorithms performance is not included. The authors of [16] propose a multi-agent DRL framework that assigns radio resources according to the slice service requirements without over-provisioning the available RAN spectrum. Differently from this work, we assume that the number of radio resources required by each network slices is already provided and we instead address the problem of multiplexing mixed-numerology spectrum slices on a shared physical layer by actively modeling the wireless channel behavior as well as the INI within the agent formulation.…”

“…In (16), {θ } corresponds to the weights of a second DNN that is used to stabilize the Q-function computation convergence and it is updated as {θ = θ} every few time steps. Parameter B is the size of the mini-batch that is randomly sampled from the experience-replay buffer.…”

Machine learning-assisted cross-slice radio resource optimization: Implementation framework and algorithmic solution

“…bandwidth) among slices using OpenFlow, being a general approach not particularized to the specificities of radio resource allocation. Recently, deep Q learning has become a quite popular tool for allocating radio resources to slices, as reflected by works [27]- [33] that include different variants of this technique and address the problem from different perspectives, such as the joint allocation of computational resources and radio resources to users in [27], the allocation of aggregate capacity per slice to multiple cells in [28], [29], the allocation of resources to slices on a single cell basis in [30], [31], [32], or the allocation of per-cell resources to the different slices jointly considering multiple cells in [33]. Finally, other works have proposed the use of traffic forecasting for cross-slice resource allocation, applying techniques such as LSTM neural networks [34], deep convolutional neural networks [35], Generative Adversarial Networks (GANs) [36], or deep neural networks [37].…”

“…the total amount of capacity to be provided to each slice. Instead, other approaches such as [28]- [32] just consider the SLA specified in terms of the QoS parameters defined at the user level, but without enforcing any aggregate capacity per slice.…”

“…In contrast to single agent solutions like those of [30], [31], which jointly consider all the tenants when making decisions, the multi-agent approach has advantages such as better scalability as it allows easily adding/removing slices in the scenario simply by adding/removing the corresponding agent. Moreover, while some multi-agent approaches have already been considered in [28], [29][32], the one considered here has the advantage that an agent learns the policy for assigning capacity to be provided to the slice in each cell, in contrast to [32], which considered the capacity in a single cell, or [28], [29], which provided the aggregated capacity over all the cells.…”