No abstract

It is well known that virtual network (VN) embedding (VNE) aims to solve how to efficiently allocate physical resources to a VN. However, this issue has been proved to be an NP-hard problem. Besides, as most of the existing approaches are based on heuristic algorithms, which is easy to fall into local optimal. To address the challenge, we formalize the problem as a mixed integer programming problem and propose a novel VNE method based on reinforcement learning in this article. And to solve the problem, we introduce a pointer network to generate virtual node mapping strategies through an attention mechanism, and design a reward function related to link resource consumption to build the connection between node mapping and link mapping stages of VNE. In addition, we present a policy gradient optimization mechanism to leverage the reward information obtained from the sampled solutions, and design an active search based process to automatically update the parameters of the neural network and to obtain near-optimal embedding solution. The experimental results show that the proposed method can improve the performance in average physical node utilization and long-term revenue to cost ratio comparing than that of the existing models. K E Y W O R D S attention, pointer network, policy gradient, reinforcement learning, virtual network embedding 1 INTRODUCTION Network virtualization technology can allow multiple heterogeneous network architectures to coexist on a shared physical substrate network (SN). 1 In network virtualization, the role of traditional Internet service providers (ISPs) is separated into two independent parts: infrastructure providers (InPs), who manage the physical infrastructure, and service providers (SPs), who provide services by renting virtual network (VN) using resources from InPs. 2 In such a way, enterprises are increasingly deploying services 3 to the Cloud with the development of network function virtualization (NFV) 4 and software defined networking (SDN). 5 The major challenges of network virtualization are how to effectively allocate SN resources to VNs as well as increase the utilization and revenue of InPs, which is known as the virtual network embedding (VNE) problem. 6 The VNE problem mainly includes two parts: node mapping stage and link mapping stage. It has been proven to be NP-hard. 7 Many exact and heuristic algorithms 8-13 have been proposed to solve the VNE problem in recent years. Although exact solutions can achieve high resource utilization rate of the SN, they require exponential time-consuming and therefore are not practical. In addition, most of the heuristic algorithms for VNE consider link mapping and node mapping separately. They did not consider the inner practical connection between the two stages and may result in suboptimal solutions. Furthermore, they are unable to leverage historical experience to optimize themselves automatically.

No abstract

No abstract