Dynamic Scheduler Management Using Deep Learning

Abstract

The ability to manage the distributed functionality of large multi-vendor networks will be an important step towards ultra-dense 5G networks. Managing distributed scheduling functionality is particularly important, due to its influence over inter-cell interference and the lack of standardization for schedulers. In this paper, we formulate a method of managing distributed scheduling methods across a small cluster of cells by dynamically selecting schedulers to be implemented at each cell. We use deep reinforcement learning methods to identify suitable joint scheduling policies, based on the current state of the network observed from data already available in the RAN. Additionally, we also explore three methods of training the deep reinforcement learning based dynamic scheduler selection system. We compare the performance of these training methods in a simulated environment against each other, as well as homogeneous scheduler deployment scenarios, where each cell in the network uses the same type of scheduler. We show that, by using deep reinforcement learning, the dynamic scheduler selection system is able to identify scheduler distributions that increase the number of users that achieve their quality of service requirements in up to 77% of the simulated scenarios when compared to homogeneous scheduler deployment scenarios.

A Novel Fault-Tolerant Aware Task Scheduler Using Deep Reinforcement Learning in Cloud Computing

“…Methodology Parameters [17] RL to make a scheduling model for cloud computing Makespan, total power cost in datacenters, energy consumption, migration time [18] RL task scheduling with Q-learning Makespan, total power cost, energy consumption [19] Task scheduling optimization algorithm called MPSO Makespan, energy consumption, packet delivery ratio, trust value [20] FELARE Ontime task completion rate, energy saving [21] RLFTWS Makespan, resource usage [22] WBCNF Computation time of tasks [23] BRCH-GWO Makespan [24] RELIEF Communication delay, reliability [25] PSO-based multipurpose algorithm Turnaround time, makespan [26] Regressive WO algorithm Processing cost, load balancing tasks [27] GO Makespan, resource utilization [28] Dynamic task scheduling algorithm based on an improved GA Total execution time and resource utilization ratio [29] PSO based on an AC algorithm Task completion time, makespan [30] DRL-based task scheduling Makespan, computation time [31] MRLCC is an approach for organizing tasks that are based on Meta RL Energy consumption, total cost, makespan [32] A novel DRL-based framework Cost and throughput, makespan [33] RLFTWS Execution time, degree of imbalance [15] DRL model Response time, makespan, CPU utilization [34] Deep reinforcement learning with PPSO SLA violation, makespan [35] DRL Cloud is an NDR-learning-based RP and TS system Estimated completion time, resource utilization [36] Deep Q-network model Degree of imbalance, cost, makespan [37] SDM reinforcement learning Energy consumption, resource utilization [38] DRLHCE Response time, degree of imbalance [39] DQN Makespan, total cost [40] Reinforcement learning Makespan [41] DDDQN-TS Task response time [31] Q-learning Makespan…”

Deep Learning for B5G Open Radio Access Network: Evolution, Survey, Case Studies, and Challenges

“…Thus, these works did not address, or include, the O-RAN architecture in their studies. In addition, a wide range of DL-based studies have also been proposed to deal with the main RAN challenges in 4G/5G networks [26][27] [28] [29]. However, these studies did not also consider the emerged O-RAN architecture, and hence need to be mapped/integrated into this architecture.…”

“…In addition, inside each PW, RL based on asynchronous advantage actor-critic (A3C) algorithm is used to perform online resource scheduling. In [29], the authors addressed the distributed scheduling challenge in order to deal with inter-cell interference and with the lack of standardization for schedulers. They proposed a reinforcement deep learning-based (RL) approach to dynamically select the suitable scheduler to each cluster of small cells, based on the channel quality and QoS constraints of the users.…”