An Attention Based Deep Reinforcement Learning Method for Virtual Network Function Placement

Li, Shuopeng; Zhang, Shaohui; Chen, Limin; Chen, Huamin; Gao, Liang; Lin, Shaofu

doi:10.1109/iccc51575.2020.9345041

“…Multi-objective approaches [19][20][21][22][23][24][25][26][27]31] solve the VNF placement problem considering multiple, and sometimes conflicting, environment aspects. For instance, the authors in [31] demonstrated the gain of VNF sharing-based service function chaining (SFC) requests placement, as a way of satisfying more requests with average-less resources per request.…”

Section: Related Workmentioning

confidence: 99%

“…Methods derived from Greedy approaches are also widely adopted to solve the VNF placement problem [17,25,27,28]. Such approaches quickly converge to a solution but tend to fall into local minimum or maximum.…”

Section: Related Workmentioning

confidence: 99%

A Novel Strategy for VNF Placement in Edge Computing Environments

Battisti

¹

,

Macedo

²

,

Josué

³

et al. 2022

View full text Add to dashboard Cite

Network function virtualization (NFV) is a novel technology that virtualizes computing, network, and storage resources to decouple the network functions from the underlying hardware, thus allowing the software implementation of such functions to run on commodity hardware. By doing this, NFV provides the necessary flexibility to enable agile, cost-effective, and on-demand service delivery models combined with automated management. Different management and orchestration challenges arise in such virtualized and distributed environments. A major challenge in the selection of the most suitable edge nodes is that of deploying virtual network functions (VNFs) to meet requests from multiple users. This article addresses the VNF placement problem by providing a novel integer linear programming (ILP) optimization model and a novel VNF placement algorithm. In our definition, the multi-objective optimization problem aims to (i) minimize the energy consumption in the edge nodes; (ii) minimize the total latency; and (iii) reducing the total cost of the infrastructure. Our new solution formulates the VNF placement problem by taking these three objectives into account simultaneously. In addition, the novel VNF placement algorithm leverages VNF sharing, which reuses VNF instances already placed to potentially reduce computational resource usage. Such a feature is still little explored in the community. Through simulation, numerical results show that our approach can perform better than other approaches found in the literature regarding resource consumption and the number of SFC requests met.

show abstract

“…The attention mechanism enables to focus on neighbor nodes with sufficient resources and contributes to the generation of neighbor interaction behaviors. Li et al [45] make a preliminary attempt to solve the VNF placement problem with the combination of attention based sequence model and RL algorithm, where the RL agent incorporates an entropy maximization strategy and the goal is formalized as optimizing the power consumption of the service chain. However, [45] assumes that the problem model is only for a star topology with 10 nodes, which is not always the case in practice.…”

Section: Related Workmentioning

confidence: 99%

“…Li et al [45] make a preliminary attempt to solve the VNF placement problem with the combination of attention based sequence model and RL algorithm, where the RL agent incorporates an entropy maximization strategy and the goal is formalized as optimizing the power consumption of the service chain. However, [45] assumes that the problem model is only for a star topology with 10 nodes, which is not always the case in practice. Our proposed A-DDPG solves the VNF-PR problem by applying the attention mechanism to the DRL architecture, using the Actor-Critic network structure.…”

Section: Related Workmentioning

confidence: 99%

A-DDPG: Attention Mechanism-based Deep Reinforcement Learning for NFV

He

¹

,

Yang

²

,

Liu

³

et al. 2021

2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS)

17

0

View full text Add to dashboard Cite

The efficacy of Network Function Virtualization (NFV) depends critically on (1) where the virtual network functions (VNFs) are placed and (2) how the traffic is routed. Unfortunately, these aspects are not easily optimized, especially under time-varying network states with different quality of service (QoS) requirements. Given the importance of NFV, many approaches have been proposed to solve the VNF placement and traffic routing problem. However, those prior approaches mainly assume that the state of the network is static and known, disregarding real-time network variations. To bridge that gap, in this paper, we formulate the VNF placement and traffic routing problem as a Markov Decision Process model to capture the dynamic network state transitions. In order to jointly minimize the delay and cost of NFV providers and maximize the revenue, we devise a customized Deep Reinforcement Learning (DRL) algorithm, called A-DDPG, for VNF placement and traffic routing in a real-time network. A-DDPG uses the attention mechanism to ascertain smooth network behavior within the general framework of network utility maximization (NUM). The simulation results show that A-DDPG outperforms the state-ofthe-art in terms of network utility, delay, and cost.

show abstract

A-DDPG: Attention Mechanism-Based Deep Reinforcement Learning for NFV

Yang

¹

,

He

²

,

Li

³

et al. 2022

Resource Allocation in Network Function Virtualization

0

View full text Add to dashboard Cite

The efficacy of Network Function Virtualization (NFV) depends critically on (1) where the virtual network functions (VNFs) are placed and (2) how the traffic is routed. Unfortunately, these aspects are not easily optimized, especially under time-varying network states with different quality of service (QoS) requirements. Given the importance of NFV, many approaches have been proposed to solve the VNF placement and traffic routing problem. However, those prior approaches mainly assume that the state of the network is static and known, disregarding real-time network variations. To bridge that gap, in this paper, we formulate the VNF placement and traffic routing problem as a Markov Decision Process model to capture the dynamic network state transitions. In order to jointly minimize the delay and cost of NFV providers and maximize the revenue, we devise a customized Deep Reinforcement Learning (DRL) algorithm, called A-DDPG, for VNF placement and traffic routing in a real-time network. A-DDPG uses the attention mechanism to ascertain smooth network behavior within the general framework of network utility maximization (NUM). The simulation results show that A-DDPG outperforms the state-ofthe-art in terms of network utility, delay, and cost.

show abstract

An Attention Based Deep Reinforcement Learning Method for Virtual Network Function Placement

Cited by 6 publications

References 14 publications

A Novel Strategy for VNF Placement in Edge Computing Environments

A Novel Strategy for VNF Placement in Edge Computing Environments

A-DDPG: Attention Mechanism-based Deep Reinforcement Learning for NFV

A-DDPG: Attention Mechanism-Based Deep Reinforcement Learning for NFV

Contact Info

Product

Resources

About