Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategies

Abstract

The recent growth of emergent network applications (e.g., satellite networks, vehicular networks) is increasing the complexity of managing modern communication networks. As a result, the community proposed the Digital Twin Networks (DTN) as a key enabler of efficient network management. Network operators can leverage the DTN to perform different optimization tasks (e.g., Traffic Engineering, Network Planning).Deep Reinforcement Learning (DRL) showed a high performance when applied to solve network optimization problems. In the context of DTN, DRL can be leveraged to solve optimization problems without directly impacting the real-world network behavior. However, DRL scales poorly with the problem size and complexity. In this paper, we explore the use of Evolutionary Strategies (ES) to train DRL agents for solving a routing optimization problem. The experimental results show that ES achieved a training time speed-up of 128 and 6 for the NSFNET and GEANT2 topologies respectively.

TD3-based Stochastic Workload Offloading for 5G-based Cloud Control System

“…Deep reinforcement learning (DRL) is an emerging technique that showed high performance when applied on solving the task offloading problem. DRL differs from traditioal optimization solutions because it leverages the knowledge learned in past optimizations, which is beneficial in highly dynamic and time-varying scenarios Güemes-Palau et al (2022). Some existing articles have used DRL to optimize task offloading in IIoT networks, medical networks and vehicle networks.…”