Network Function Virtualization (NFV) and service orchestration simplify the deployment and management of network and telecommunication services. The deployment of these services requires, typically, the allocation of Virtual Network Function -Forwarding Graph (VNF-FG), which implies not only the fulfillment of the service's requirements in terms of Quality of Service (QoS), but also considering the constraints of the underlying infrastructure. This topic has been well-studied in existing literature, however, its complexity and uncertainty of available information unveil challenges for researchers and engineers. In this paper, we explore the potential of reinforcement learning techniques for the placement of VNF-FGs. However, it turns out that even the most well-known learning technique is ineffective in the context of a large-scale action space. In this respect, we propose approaches to find out feasible solutions while improving significantly the exploration of the action space. The simulation results clearly show the effectiveness of the proposed learning approach for this category of problems. Moreover, thanks to the deep learning process, the performance of the proposed approach is improved over time.

Knowledge-Defined networking (KDN) is a concept that relies on Software-Defined networking (SDN) and Machine Learning (ML) in order to operate and optimize data networks. Thanks to SDN, a centralized path calculation can be deployed, thus enhancing the network utilization as well as Quality of Services (QoS). QoS-aware routing problem is a high complexity problem, especially when there are multiple flows coexisting in the same network. Deep Reinforcement Learning (DRL) is an emerging technique that is able to cope with such complex problem. Recent studies confirm the ability of DRL in solving complex routing problems; however, its performance in the network with QoSsensitive flows has not been addressed. In this paper, we exploit a DRL agent with convolutional neural networks in the context of KDN in order to enhance the performance of QoS-aware routing. The obtained results demonstrate that the proposed approach is able to improve the performance of routing configurations significantly even in complex networks.

The placement of Virtual Network Function -Forwarding Graphs (VNF-FGs) is one of the basic operations in the networks of the future. Being NP-hard, several heuristics and metaheuristics have been proposed. However, these approaches are inefficient due to the need to recalculate the solution at each service placement. In this paper, we adapt one of the most advanced approaches in Deep Reinforcement Learning (DRL), in order to improve exploration by generalizing the neural network calculating action values. We also propose an evolutionary algorithm to evolve these neural networks in order to discover better ones, which also avoids getting stuck in local minima. In order to avoid going through the almost innumerable number of infeasible solutions, we propose a heuristic, which combined with our DRL, makes it possible to guarantee the feasibility of the solutions and therefore to make the placement much more efficient. The simulation results we obtained confirm the quality of the solutions obtained as well as the superiority of the proposed solution over the existing one.

Network slicing remains one of the key technologies in 5G and beyond 5G networks (B5G). By leveraging SDN and NVF techniques, it enables the coexistence of several heterogeneous virtual networks (VNs) on top of the same physical infrastructure. Despite the advantages it brings to network operators, network slicing raises a major challenge: Resource allocation of VNs, also known as the virtual network embedding problem (VNEP). VNEP is known to be an NP-Hard problem. Several heuristics, meta-heuristics and Deep Reinforcement Learning (DRL) based solutions were proposed in the literature to solve it. Regarding the first two categories, they can provide a solution for large scale problems within a reasonable time, but the solution is usually suboptimal, which leads to an inefficient utilization of the resources and increases the cost of the allocation process. For DRL-based approaches and due to the explorationexploitation dilemma, the solution can be infeasible. To overcome these issues, we combine, in this work, deep reinforcement learning and relational graph convolutional neural networks in order to automatically learn how to improve the quality of VNEP heuristics. Simulation results show the effectiveness of our approach. Starting with an initial solution given by the heuristics our approach can find an amelioration, with an improvement in the order of 35%.

No abstract