Optimal Resource Allocation Considering Non-Uniform Spatial Traffic Distribution in Ultra-Dense Networks: A Multi-Agent Reinforcement Learning Approach

Abstract

Recently, the demand for small cell base stations (SBSs) is exploding to accommodate the explosive increase in mobile data traffic. In ultra-dense small cell networks (UDSCNs), because the spatial and temporal traffic distributions are significantly disproportionate, the efficient management of the energy consumption of SBSs is crucial. Therefore, we herein propose a multi-agent distributed Q-learning algorithm that maximizes energy efficiency (EE) while minimizing the number of outage users. Through intensive simulations, we demonstrate that the proposed algorithm outperforms conventional algorithms in terms of EE and the number of outage users. Furthermore, it is shown that the proposed reinforcement learning algorithm can converge to the optimal solution despite its significantly lower computational complexity than the centralized approach.INDEX TERMS Multi-agent Q-learning, spatial traffic distribution, energy efficiency, user outage, ultradense small cell network.

Intelligence-learning driven resource allocation for B5G Ultra-Dense Networks: A structured literature review

“…A Q-value approximation is proposed to reduce the state space, which results in better performance than conventional algorithms and ensures the QoS of primary and secondary UEs. Kim et al (2022a) propose a power transmission control based on QL to maximize the EE while minimizing the number of UE outages. In this scheme, the UE considers only their past action for the decisionmaking while the reward function considers the global network performance.…”

Low-Complexity Q-Learning for Energy-Aware Small-Cell Networks With Integrated Access and Backhaul

“…Fur-thermore, access and backhaul networks require a large amount of spectrum bandwidth as well as consume a vast amount of energy to support many kinds of real-time mobile application services. In particular, access technology accounts for a large portion of the total network energy consumption [6]- [8]. Also, to meet the technical requirements of…”

“…Also, the explosive installations of small cell base stations (SBSs) to support the increasing network traffic are accelerating network densification, and thus it may result in the need for more available frequency resources and an increase in network power consumption [7], [8]. Hence, network operators should efficiently utilize these limited frequency resources to support massive users within the entire network and optimally deploys many SBSs to reduce spectrum holes and network power consumption.…”