DVNE-DRL: Dynamic virtual network embedding algorithm based on deep reinforcement learning

Abstract

Virtual Network Embedding (VNE), as the key challenge of network resource management technology, lies in the contradiction between online embedding decision and pursuing long-term average revenue goals. Most of the previous work ignored the dynamics in Virtual Network (VN) modeling, or could not automatically detect the complex and time-varying network state to provide a reasonable network embedding scheme. In view of this, we model a network embedding framework where the topology and resource allocation change dynamically with the number of network users and workload, and then introduce a deep reinforcement learning method to solve the VNE problem. Further, a dynamic virtual network embedding algorithm based on Deep Reinforcement Learning (DRL), named DVNE-DRL, is proposed. In DVNE-DRL, VNE is modeled as Markov Decision Process (MDP), and Reinforcement Learning Agent (RLA) is used to periodically detect the state of network environment and provide a more reasonable network embedding scheme in real time. Different from the previous methods based on DRL algorithm to solve VNE problem, this paper improves the method of feature extraction and matrix optimization, and considers the characteristics of virtual network and physical network together to alleviate the problem of redundancy and slow convergence. The simulation results show that compared with the existing advanced algorithms, the acceptance rate and average revenue of DVNE-DRL are increased by about 25% and 35%, respectively. K E Y W O R D S VNE, DRL, embedding framework, DRL-DVNE，MDP，feature extraction