Deep Reinforcement Learning-Based Deterministic Routing and Scheduling for Mixed-Criticality Flows

Abstract

Deterministic networking (DetNet) has recently drawn much attention by investigating deterministic flow scheduling. Combined with artificial intelligent (AI) technologies, it can be leveraged as a promising network technology for facilitating automated network configuration in the Industrial Internet of Things (IIoT). However, the stricter requirements of the IIoT have posed significant challenges, that is, deterministic and bounded latency for time-critical applications. This paper incorporates deep reinforcement learning (DRL) in Cycle Specified Queuing and Forwarding (CSQF) and proposes a DRLbased Deterministic Flow Scheduler (Deep-DFS) to solve the Deterministic Flow Routing and Scheduling (DFRS) problem. Novel delay aware network representations, action masking and criticality aware reward function design are proposed to make Deep-DFS more scalable and efficient. Simulation experiments are conducted to evaluate the performances of Deep-DFS, and the results show that Deep-DFS can schedule more flows than the other benchmark methods (heuristic-based and AI-based methods).

Double Deep Q-Learning-Based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

“…In this section, the system is assumed to be fully known, i.e., the list of services and their characteristics are available, and the current state of the network and cloud resources as well as requests and their requirements are being monitored and collected on a regular basis. This could be the case of an industrial environment whereby tasks and communications among robots and devices are pre-planned [32], [33], [34]. Under such scenarios, the following section proposes two methods, B&B-CCRA and WF-CCRA, to solve the problem specified by (1).…”

Reinforcement Learning for Time-Aware Shaping (IEEE 802.1Qbv) in Time-Sensitive Networks

“…Closer to our solution, [11] uses DRL to schedule and route mixed-criticality flows in a Deterministic Networking (DetNet) environment. DetNet operates at the layer 3 whereas TSN operates at layer 2.…”

Toward 6G-Based Metaverse: Supporting Highly-Dynamic Deterministic Multi-User Extended Reality Services

“…In contrast to displaying images or playing movies, user interaction, for example, waving a hand right or left to switch content, pressing a finger forward to push a "button" and saying "close" to stop the application, requires real-time communications between user/user or user/server. Therefore, the deterministic networking (DetNet) [3] capabilities, which enable deterministic data delivery, processing and synchronization capacity provided by the underlying networks, become necessary to support the high-quality immersive user experience in real-time.…”