Deep Reinforcement Learning for Resource Management in Network Slicing

Li, Rongpeng; Zhao, Zhifeng; Sun, Qi; Chih‐Lin, I; Yang, Chenyang; Chen, Xianfu; Zhao, Minjian; Zhang, Honggang

doi:10.1109/access.2018.2881964

Cited by 295 publications

(232 citation statements)

References 22 publications

Supporting

Mentioning

228

Contrasting

Unclassified

Order By: Relevance

“…Deep RL has been found effective in network slicing [64], integrated design of caching, computing, and communication for software-defined and virtualized vehicular networks [65], multi-tenant cross-slice resource orchestration in cellular RANs [66], proactive channel selection for LTE in unlicensed spectrum [67], and beam selection in millimeter wave MIMO systems in [68]. In this section, we highlight some exemplary cases where deep RL shows impressive promises in wireless resource allocation, in particular, for dynamic spectrum access, power allocation, and joint spectrum and power allocation in vehicular networks.…”

Section: Deep Reinforcement Learning Based Resource Allocationmentioning

confidence: 99%

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

et al. 2020

View full text Add to dashboard Cite

It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, e.g., in the highmobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this paper, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep learning assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.

show abstract

Section: Deep Reinforcement Learning Based Resource Allocationmentioning

confidence: 99%

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…As a non-nascent algorithm, RL has been widely applied in the field of cognitive radio [6] and green communications [7]. The recent well-known application success in Go [8] further proves the feasibility to utilize neural networks (NN) to approximate the value functions in classical RL with case-testified convergence stability, and triggers tremendous research attention in the communications and networking area to solve resource allocation issues in some specific fields like power control, green communications, cloud radio access networks, mobile edge computing and caching [3]. But, a common problem in these works is that researchers usually assume a rather limited small discrete action space to ensure the necessary convergence rate.…”

Section: Introductionmentioning

confidence: 99%

“…But, a common problem in these works is that researchers usually assume a rather limited small discrete action space to ensure the necessary convergence rate. For example, [3] realized the spectrum allocation per slice on the unit of MegaHertz and accordingly design a DRL framework with tens of possible actions. But, such a coarse-grained spectrum allocation solution inevitably decreases the SE when some slice has very few service activities.…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning With Discrete Normalized Advantage Functions for Resource Management in Network Slicing

Chen

Hua

et al. 2019

IEEE Commun. Lett.

Self Cite

View full text Add to dashboard Cite

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep Q-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the varying demands and the allocated bandwidth as the environment state and the action, respectively. However, allocating bandwidth in a finer resolution usually implies larger action space, and unfortunately DQL fails to quickly converge in this case. In this paper, we introduce discrete normalized advantage functions (DNAF) into DQL, by separating the Qvalue function as a state-value function term and an advantage term and exploiting a deterministic policy gradient descent (DPGD) algorithm to avoid the unnecessary calculation of Qvalue for every state-action pair. Furthermore, as DPGD only works in continuous action space, we embed a k-nearest neighbor algorithm into DQL to quickly find a valid action in the discrete space nearest to the DPGD output. Finally, we verify the faster convergence of the DNAF-based DQL through extensive simulations.

show abstract

“…However, in order to provide better-performing and costefficient services, RAN slicing involves more challenging technical issues for the realtime resource management on existing slices, since (a) for radio access networks, spectrum is a scarce resource and it is essential to guarantee the spectrum efficiency (SE) [7]; (b) the service level agreements (SLAs) with slice tenants usually impose stringent requirements on quality of experience (QoE) perceived by users; and (c) the actual demand of each slice heavily depends on the request patterns of mobile users [11]. Therefore, the classical dedicated resource allocation fails to simultaneously address these problems [9].…”

Section: Introductionmentioning

confidence: 99%

“…Instead, it is necessary to intelligently allocate the spectrum to slices according to the dynamics of service request from mobile users coherently [12], so as to obtain satisfactory QoE in each slice at the cost of acceptable SE. There have been a number of research works towards this intelligent resource management for network slicing [11], [13]- [16]. In particular, [13] proposed an online genetic slicing strategy optimizer for inter-slice resource management.…”

Section: Introductionmentioning

confidence: 99%

GAN-Based Deep Distributional Reinforcement Learning for Resource Management in Network Slicing

Hua

Zhao

et al. 2019

2019 IEEE Global Communications Conference (GLOBECOM)

Self Cite

View full text Add to dashboard Cite

Network slicing is a key technology in 5G communications system, which aims to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demandaware resource allocation is of significant importance to network slicing. In this paper, we consider a scenario that contains several slices in radio access networks with base stations sharing the same bandwidth. Deep reinforcement learning (DRL) is leveraged to solve this problem by regarding the varying service demands and the allocated bandwidth as the environment state and action, respectively. In order to tackle the annoying randomness and noise embedded in the received quality of experience (QoE) satisfaction ratio and spectrum efficiency (SE), we propose generative adversarial network (GAN) based deep distributional Q network (GAN-DDQN) to learn the distribution of action values. Furthermore, we estimate the distributions by approximating a full quantile function, so as to make the training error more controllable. For the sake of protecting the stability of GAN-DDQN's training process from the widely-spanning utility values, we also put forward a reward-clipping mechanism. Finally, we verify the performance of the proposed GAN-DDQN algorithm through extensive simulations.Index Terms-network slicing, deep reinforcement learning, distributional reinforcement learning, generative adversarial network, 5G

show abstract

Deep Reinforcement Learning for Resource Management in Network Slicing

Cited by 295 publications

References 22 publications

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep Reinforcement Learning With Discrete Normalized Advantage Functions for Resource Management in Network Slicing

GAN-Based Deep Distributional Reinforcement Learning for Resource Management in Network Slicing

Contact Info

Product

Resources

About