QoE-driven Antenna Tuning in Cellular Networks With Cooperative Multi-agent Reinforcement Learning

Liu, Xuewen; Chuai, Gang; Wang, Xin; Xu, Zhiwei; Gao, Weidong; Zhang, Kaisa; Liu, Qian; Maimaiti, Saidiwaerdi; Zuo, Peiliang

doi:10.1109/tmc.2022.3230711

Cited by 4 publications

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above works modeled the policy search process as a Markov decision process, which is true if different agents update their policies independently at different times. Nevertheless, if two or more agents update their policies at the same time, a non-stationary multi-agent environment may occur [44]. How to reduce the action space and computational complexity of multi-agent systems to improve the training speed while ensuring a stationary multi-agent environment is a key issue.…”

Section: Related Workmentioning

confidence: 99%

Joint Resource Allocation and Drones Relay Selection for Large-Scale D2D Communication Underlaying Hybrid VLC/RF IoT Systems

Liu,

Huang,

Zhang

et al. 2023

Drones

Self Cite

View full text Add to dashboard Cite

Relay-aided Device-to-Device (D2D) communication combining visible light communication (VLC) with radio frequency (RF) is a promising paradigm in the internet of things (IoT). Static relay limits the flexibility and maintaining connectivity of relays in Hybrid VLC/RF IoT systems. By using a drone as a relay station, it is possible to avoid obstacles such as buildings and to communicate in a line-of-sight (LoS) environment, which naturally aligns with the requirement of VLC Systems. To further support the application of VLC in the IoT, subject to the challenges imposed by the constrained coverage, the lack of flexibility, poor reliability, and connectivity, drone relay-aided D2D communication appears on the horizon and can be cost-effectively deployed for the large-scale IoT. This paper proposes a joint resource allocation and drones relay selection scheme, aiming to maximize the D2D system sum rate while ensuring the quality of service (QoS) requirements for cellular users (CUs) and D2D users (DUs). First, we construct a two-phase coalitional game to tackle the resource allocation problem, which exploits the combination of VLC and RF, as well as incorporates a greedy strategy. After that, a distributed cooperative multi-agent reinforcement learning (MARL) algorithm, called WoLF policy hill-climbing (WoLF-PHC), is proposed to address the drones relay selection problem. Moreover, to further reduce the computational complexity, we propose a lightweight neighbor–agent-based WoLF-PHC algorithm, which only utilizes historical information of neighboring DUs. Finally, we provide an in-depth theoretical analysis of the proposed schemes in terms of complexity and signaling overhead. Simulation results illustrate that the proposed schemes can effectively improve the system performance in terms of the sum rate and outage probability with respect to other outstanding algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

Joint Resource Allocation and Drones Relay Selection for Large-Scale D2D Communication Underlaying Hybrid VLC/RF IoT Systems

Liu,

Huang,

Zhang

et al. 2023

Drones

Self Cite

View full text Add to dashboard Cite

show abstract

A Differentiable Throughput Model for Load-Aware Cellular Network Optimization Through Gradient Descent

Eller,

Svoboda,

Rupp

2024

IEEE Access

View full text Add to dashboard Cite

The efficient operation of cellular networks requires careful tuning of configuration parameters, such as the transmit power or antenna tilts, to adequately balance interference while providing the necessary capacity to the connected UEs. As manual tuning of these parameters is typically unfeasible, several automated coverage and capacity optimization methods have been proposed. However, most existing solutions are either based on poorly scalable black-box optimization methods or solely consider interference management, while omitting the potential of congested cells. In this work, we instead propose a differentiable framework for cellular network optimization, centered around the end-user throughput, that enables load-aware tuning of network parameters through gradient descent. Hereby, we approach the problem from a data-driven perspective, and include dedicated model subcomponents derived from monitoring data, which enable the calibration to site-specific traffic patterns and KPI measurements. We validate our approach for joint transmit power optimization in a real-world network layout with ≈ 150 cells in two frequency bands. In our evaluation, the gradient descent-based optimization reliably reduces the outage ratio for different levels of demand, while the black-box baseline struggles to explore the large search space. Our results further reveal substantial differences between the proposed load-aware and commonly used SINR-based objectives, for which we repeatably obtain unbalanced network configurations with severely congested cells. In contrast, the proposed end-user throughput objective promotes a balanced network configuration, providing adequate resources to the connected UEs while also limiting interference.

show abstract

Q-Learning Inspired Method for Antenna Azimuth Selection in Cellular Networks

Chmieliauskas,

Mahmood,

Thar

et al. 2023

2023 Workshop on Microwave Theory and Technology in Wireless Communications (MTTW)

View full text Add to dashboard Cite

QoE-driven Antenna Tuning in Cellular Networks With Cooperative Multi-agent Reinforcement Learning

Cited by 4 publications

References 28 publications

Joint Resource Allocation and Drones Relay Selection for Large-Scale D2D Communication Underlaying Hybrid VLC/RF IoT Systems

Joint Resource Allocation and Drones Relay Selection for Large-Scale D2D Communication Underlaying Hybrid VLC/RF IoT Systems

A Differentiable Throughput Model for Load-Aware Cellular Network Optimization Through Gradient Descent

Q-Learning Inspired Method for Antenna Azimuth Selection in Cellular Networks

Contact Info

Product

Resources

About