Network Slice Reconfiguration by Exploiting Deep Reinforcement Learning With Large Action Space

Abstract

It is widely acknowledged that network slicing can tackle the diverse usage scenarios and connectivity services that the 5G-and-beyond system needs to support. To guarantee performance isolation while maximizing network resource utilization under dynamic traffic load, network slice needs to be reconfigured adaptively. However, it is commonly believed that the finegrained resource reconfiguration problem is intractable due to the extremely high computational complexity caused by numerous variables. In this paper, we investigate the reconfiguration within a core network slice with aim of minimizing long-term resource consumption by exploiting Deep Reinforcement Learning (DRL). This problem is also intractable by using conventional Deep Q Network (DQN), as it has a multi-dimensional discrete action space which is difficult to explore efficiently. To address the curse of dimensionality, we propose a discrete Branching Dueling Qnetwork (discrete BDQ) by incorporating the action branching architecture into DQN, for drastically decreasing the number of estimated actions. Based on the discrete BDQ network, we develop an intelligent network slice reconfiguration algorithm (INSRA). Extensive simulation experiments are conducted to evaluate the performance of INSRA and the numerical results reveal that INSRA can minimize the long-term resource consumption and achieve high resource efficiency compared with several benchmark algorithms.

Hybrid Model-Data Driven Network Slice Reconfiguration by Exploiting Prediction Interval and Robust Optimization

“…Data-driven approaches use data as input to make end-toend slice reconfiguration decisions [5], [18]- [20]. The authors of [18] proposed an intelligent resource scheduling strategy by exploiting a collaborative learning framework that consists of Deep Learning (DL) in conjunction with Reinforcement Learning (RL).…”

“…The reconfiguration of network slices can be either intra-slice reconfiguration or inter-slice reconfiguration. Intra-slice reconfiguration is performed at small time scales which adjusts the flow path and the association of VNF instance of the flows within individual network slices [5], while inter-slice reconfigurations are performed at large time scales which involves resource scaling between multiple network slices(i.e., slice breathing [6]) and VNF migration (i.e., slice mobility [7]). Some recent work such as [4], [8] propose to adaptively reconfigure network slices to meet users' instantaneous traffic demands.…”

AI Meets CRNs: A Prospective Review on the Application of Deep Architectures in Spectrum Management

“…With the concept of performance isolation between network slices requiring fine-grained resource reconfiguration, which leads to extremely high computational complexity. which affects the operations of all the different layers of the protocol stack, a problem which was revisited in [169]. Here, the authors investigated the finegrained reconfiguration within the core network slice with the objective of minimizing long-term resource consumption.…”

Proactive Network Slice Reconfiguration by Exploiting Prediction Interval and Robust Optimization

“…Furthermore, the variables {z p,q (e k m )} and {x i,k (f k m )} should be constrained to form a connected path for each network slice [11]. Therefore, for all…”

“…The topology of the substrate network is the same as that in [11], which is widely used in network slicing simulations. The VNF set of the system contains 7 VNFs, and the compression ratio of each VNF is uniformly distributed in [0.8, 1.2].…”