A Reinforcement Learning Framework for QoS-Driven Radio Resource Scheduler

Open Radio Access Network (O-RAN) is a novel architecture that enables the disaggregation and the virtualization of network components. This would provide new ways to mix and match network components by "opening up" the interfaces between them. O-RAN enables driving down the costs of network deployments and allows the entry of new players into the RAN market. It enables network operators to maximize resource utilization and deliver new network edge services at a lower cost, resulting in higher profits for operators. In this context, we consider a computing resource allocation problem for maximizing the operator's profit. Given that an operator receives subscribers' payments and pays the infrastructure provider's costs, we model the problem using Mixed Integer Linear Programming (MILP). Then, we propose to solve the problem using Reinforcement Learning (RL). Our simulation results demonstrate the ability of the RL agent to increase the operator's profit while reducing the algorithmic complexity of the MILP solver.

show abstract

“…This will make the size of the NN huge and penalize the convergence time. As in [16] and [17], we use the dynamic architecture shown in Fig. 2…”

Section: Reinforcement Learning Modelmentioning

confidence: 99%

Reinforcement Learning based model for Maximizing Operator's Profit in Open-RAN

Sharara

Hoteit

2023

NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium

View full text Add to dashboard Cite

show abstract

“…The proposed architecture minimizes the neural network's size, reducing the computational requirements. This flexible architecture was also used in [23]. While their RL agent takes just one action per TTI, ours takes one action in a step, and each episode is the collection of decisions made at each step (the selection of a user at each step) in one TTI.…”

Section: Rl Architecturementioning

confidence: 99%

Policy-Gradient-Based Reinforcement Learning for Computing Resources Allocation in O-RAN

Sharara

Pamuklu

Hoteit

et al. 2022

2022 IEEE 11th International Conference on Cloud Networking (CloudNet)

View full text Add to dashboard Cite

Open Radio Access Network (O-RAN) is a novel architecture aiming to disaggregate the network components to reduce capital and operational costs and open the interfaces to ensure interoperability. In this work, we consider the problem of allocating computing resources to process the data of enhanced Mobile BroadBand (eMBB) users and Ultra-Reliable Low-Latency (URLLC) Users. Supposing the processing of users' frames from different base stations is done in a shared O-Cloud, we model the computing resources allocation problem as an Integer Linear Programming (ILP) problem that aims at fairly allocating computing resources to eMBB and URLLC users and optimizing the QoS of URLLC users without neglecting eMBB users. Due to the high complexity of solving an ILP problem, we model the problem using Reinforcement Learning (RL). Our results demonstrate the ability of our RL-based solution to perform close to the ILP solver while having much lower computational complexity. For a different number of Open Radio Units (O-RUs), the objective value of the RL agent does not deviate from the ILP objective by more than 6%.

show abstract

“…The action changes the state in which a reward can be calculated. The RL state consists of information from both channels and queues, generically denoted as CSI and queue state information (QSI), respectively, similar to [54]. Thus, the scheduler is cross-layer since it considers information from layers other than the PHY, such as the buffer occupancy of active users and the age of the packets.…”

Section: A States Actions and Rewardsmentioning

confidence: 99%

Deep Reinforcement Learning-Based Scheduling for Multiband Massive MIMO

Lopes

Nahum²,

Dreifuerst

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Fifth-generation (5G) cellular communication systems have embraced massive multipleinput-multiple-output (MIMO) in the low-and mid-band frequencies. In a multiband system, the base station can serve different users in each band, while the user equipment can operate only in a single band simultaneously. This paper considers a massive MIMO system where channels are dynamically allocated in different frequency bands. We treat multiband massive MIMO as a scheduling and resource allocation problem and propose deep reinforcement learning (DRL) agents to perform user scheduling. The DRL agents use buffer and channel information to compose their observation space, and the agent's reward function maximizes the transmitted throughput and minimizes the packet loss rate. We compare the proposed DRL algorithms with traditional baselines, such as maximum throughput and proportional fairness. The results show that the DRL models outperformed baselines obtaining a 20% higher network sum rate and an 84% smaller packet loss rate. Moreover, we compare different DRL algorithms focusing on training time to assess the online implementation of the DRL agents, showing that the best agent needs about 50K training steps to converge.

show abstract

A Reinforcement Learning Framework for QoS-Driven Radio Resource Scheduler

Cited by 11 publications

References 11 publications

Reinforcement Learning based model for Maximizing Operator's Profit in Open-RAN

Reinforcement Learning based model for Maximizing Operator's Profit in Open-RAN

Policy-Gradient-Based Reinforcement Learning for Computing Resources Allocation in O-RAN

Deep Reinforcement Learning-Based Scheduling for Multiband Massive MIMO

Contact Info

Product

Resources

About