Elastic O-RAN Slicing for Industrial Monitoring and Control: A Distributed Matching Game and Deep Reinforcement Learning Approach

Abstract

In this work, we design an elastic open radio access network (O-RAN) slicing for the industrial Internet of things (IIoT). Unlike IoT, IIoT poses additional challenges such as severe communication environment, network-slice resource demand variations, and on-time information update from the IIoT devices during industrial production. First, we formulate the O-RAN slicing problem for on-time industrial monitoring and control where the objective is to minimize the cost of fresh information updates (i.e., age of information (AoI)) from the IIoT devices (i.e., sensors) while maintaining the energy consumption of those devices with the energy constraint as well as O-RAN slice isolation constraints. Second, we propose the intelligent ORAN framework based on game theory and machine learning to mitigate the problem’s complexity. We propose a two-sided distributed matching game in the O-RAN control layer that captures the IIoT channel characteristics and the IIoT service priorities to create IIoT device and small cell base station (SBS) preference lists. We then employ an actor-critic model with a deep deterministic policy gradient (DDPG) in the O-RAN service management layer to solve the resource allocation problem for optimizing the network slice configuration policy under time varying slicing demand. While the matching game helps the actor-critic model, the DDPG enforces the long-term policy-based guidance for resource allocation that reflects the trends of all IIoT devices and SBSs satisfactions with the assignment. Finally, the simulation results show that the proposed solution enhances the performance gain for the IIoT services by serving an average of 50% and 43.64% more IIoT devices than the baseline approaches. <br>

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

“…In Table 10, we categorize the reviewed proposals based on the ML technique and the type and number of used algorithms. According to our analysis of the literature review on resource management in RAN slicing, as shown in T-NN 1 Evolutionary Algorithms [65], [66], [68], [69], [67], [70], [72], [71] GA 8 [35], [79], [80], [81], [129], [130], [95] AC 7 [78] MAB 1 [111], [85], [117], [86], [87], [88], [136], [90], [96], [97], [98], [151], [126], [99], [123], [131], [114], [118], [94], [100], [103], [119], [104], [105] DQN 25 [109], [108] Q-Learning 2 Reinforcement Learning [115], [112], [84] Double DQN 3 …”

“…The simulation results demonstrate that the Spectrum Efficiency in DRL is higher than the proportionally fair, water-filling, and round-robin allocation methods for eMBB, uRLLC, and mMTC slices. The authors of [95] have proposed an elastic method based on distributed game theory and RL to allocate resources in a macro-cell BS within O-RAN to meet the needs of three different classes of Industrial IoT. In this approach, the authors aimed to balance the age of information (AoI) and energy efficiency while maximizing service rates using synchronization between multiple devices.…”

“…In the reviewed methods for O-RAN (e.g., [94], [100], [124]), NN have been used to overcome the size of Q-Table in Q-learning. To increase decision stability in AC [95], we can use a memory buffer to eliminate inefficient decisions or use A2C like [124].…”

“…In recent years, the use of distributed methods (such as transfer learning and federated learning) in ML has been considered due to the common challenges in centralized methods [79], [80], [81], [82], [84], [85], [86], [87], [88], [89], [90], [91], [92], [83], [93], [94], [95], [110], [111], [112], [113], [114], [115], [116], [117], [118], [136] ✓ ✓ ---- [96], [97], [98], [99], [100], [103], [104], [105], [123], [124], [135] [107], [108], [109], [126], [127], [128], [35], [129], [130] such as centralization, security, privacy, and time and computational complexity. In distributed methods, a ML mode...…”

Industrial Digital Twins at the Nexus of NextG Wireless Networks and Computational Intelligence: A Survey

“…Each objective must capture the requirements and requests of industrial applications realistically using critical metrics as service rate, scheduling and isolation, information freshness, and energy efficiency. In this direction, in [151], using distributed game theory and machine learning, the authors developed an elastic network slice policy to satisfy time-varying resource allocation demands for three different industrial traffic classes. In the slice configuration policy, the authors mainly aimed to balance AoI and energy efficiency while maximizing the service rate.…”