With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by virtualizing network functions and placing them on shared commodity servers. However, one of the critical issues in NFV is the resource allocation for the highly complex services; moreover, this problem is classified as an NP-Hard problem. To solve this problem, our work investigates the potential of Deep Reinforcement Learning (DRL) as a swift yet accurate approach (as compared to integer linear programming) for deploying Virtualized Network Functions (VNFs) under several Quality-of-Service (QoS) constraints such as latency, memory, CPU, and failure recovery requirements. More importantly, the failure recovery requirements are focused on the node-outage problem where outage can be either due to a disaster or unavailability of network topology information (e.g., due to proprietary and ownership issues). In DRL, we adopt a Deep Q-Learning (DQL) based algorithm where the primary network estimates the action-value function Q, as well as the predicted Q, highly causing divergence in Q-value’s updates. This divergence increases for the larger-scale action and state-space causing inconsistency in learning, resulting in an inaccurate output. Thus, to overcome this divergence, our work has adopted a well-known approach, i.e., introducing Target Neural Networks and Experience Replay algorithms in DQL. The constructed model is simulated for two real network topologies—Netrail Topology and BtEurope Topology—with various capacities of the nodes (e.g., CPU core, VNFs per Core), links (e.g., bandwidth and latency), several VNF Forwarding Graph (VNF-FG) complexities, and different degrees of the nodal outage from 0% to 50%. We can conclude from our work that, with the increase in network density or nodal capacity or VNF-FG’s complexity, the model took extremely high computation time to execute the desirable results. Moreover, with the rise in complexity of the VNF-FG, the resources decline much faster. In terms of the nodal outage, our model provided almost 70–90% Service Acceptance Rate (SAR) even with a 50% nodal outage for certain combinations of scenarios.

The Internet of Things (IoT) universe will continue to expand with the advent of the sixth generation of mobile networks (6G), which is expected to support applications and services with higher data rates, ultra-reliability, and lower latency compared to the fifth generation of mobile networks (5G). These new demanding 6G applications will introduce heavy load and strict performance requirements on the network. Network Function Virtualization (NFV) is a promising approach to handling these challenging requirements, but it also poses significant Resource Allocation (RA) challenges. Especially since 6G network services will be highly complicated and comparatively short-lived, network operators will be compelled to deploy these services in a flexible, on-demand, and agile manner. To address the aforementioned issues, microservice approaches are being investigated, in which the services are decomposed and loosely coupled, resulting in increased deployment flexibility and modularity. This study investigates a new RA approach for microservices-based NFV for efficient deployment and decomposition of Virtual Network Function (VNF) onto substrate networks. The decomposition of VNFs involves additional overheads, which have a detrimental impact on network resources; hence, finding the right balance of when and how much decomposition to allow is critical. Thus, we develop a criterion for determining the potential/candidate VNFs for decomposition and also the granularity of such decomposition. The joint problem of decomposition and efficient embedding of microservices is challenging to model and solve using exact mathematical models. Therefore, we implemented a Reinforcement Learning (RL) model using Double Deep Q-Learning, which revealed an almost 50% more normalized Service Acceptance Rate (SAR) for the microservice approach over the monolithic deployment of VNFs.

With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by disintegrating the dependency on the hardware devices via virtualizing the network functions and placing them on shared data centres. However, one of the main challenges of the NFV paradigm is the resource allocation problem which is known as NFV-Resource Allocation (NFV-RA). NFV-RA is a method of deploying software-based network functions on the substrate nodes, subject to the constraints imposed by the underlying infrastructure and the agreed Service Level Agreement (SLA). This work investigates the potential of Reinforcement Learning (RL) as a fast yet accurate means (as compared to integer linear programming) for deploying the softwarized network functions onto substrate networks under several Quality of Service (QoS) constraints. In addition to the regular resource constraints and latency constraints, we introduced the concept of a complete outage of certain nodes in the network. This outage can be either due to a disaster or unavailability of network topology information due to proprietary and ownership issues. We have analyzed the network performance on different network topologies, different capacities of the nodes and the links, and different degrees of the nodal outage. The computational time escalated with the increase in the network density to achieve the optimal solutions; this is because Q-Learning is an iterative process which results in a slow exploration. Our results also show that for certain topologies and a certain combination of resources, we can achieve between 70-90% service acceptance rate even with a 40% nodal outage.