A Reinforcement Learning Based Low-Delay Scheduling With Adaptive Transmission

Zhao, Yu; Lee, Joohyun

doi:10.1109/ictc46691.2019.8939680

Cited by 1 publication

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate the performance of the Q-greedyUCB algorithm, we implement a MATLAB simulation with different input parameters. There are two cases: i) change the maximum buffer size B, the number of packets in each data arrival M , and the maximum number of transmitted packets in each time slot C under the condition of constant arrival rate α. ii) change the parameters α under the condition of constant B, M , and C. Additionally, we compare the Q-greedyUCB algorithm with Policy Iteration (PI) in [3], the Q-learning algorithm in [26] and the Average-payoff RL algorithm (ARL) in [27].…”

Section: Simulation Resultsmentioning

confidence: 99%

Q-greedyUCB: a New Exploration Policy for Adaptive and Resource-efficient Scheduling

Zhao,

Lee,

Chen

2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a learning algorithm to find a scheduling policy that achieves an optimal delay-power trade-off in communication systems. Reinforcement learning (RL) is used to minimize the expected latency for a given energy constraint where the environments such as traffic arrival rates or channel conditions can change over time. For this purpose, this problem is formulated as an infinite-horizon Markov Decision Process (MDP) with constraints. To handle the constrained optimization problem, we adopt the Lagrangian relaxation technique to solve it. Then, we propose a variant of Q-learning, Q-greedyUCB that combines Q-learning for average reward algorithm and Upper Confidence Bound (UCB) policy to solve this decision-making problem. We prove that the Q-greedyUCB algorithm is convergent through mathematical analysis. Simulation results show that Q-greedyUCB finds an optimal scheduling strategy, and is more efficient than Q-learning with the ε-greedy and Average-payoff RL algorithm in terms of the cumulative reward (i.e., the weighted sum of delay and energy) and the convergence speed. We also show that our algorithm can reduce the regret by up to 12% compared to the Q-learning with the ε-greedy and Average-payoff RL algorithm.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%