User Association in a VHetNet With Delayed CSI: A Deep Reinforcement Learning Approach

Khoshkbari, Hesam; Sharifi, Sara; Kaddoum, Georges

doi:10.1109/lcomm.2023.3291613

Cited by 11 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The formulated problem is solved using a combination of integer linear programming and generalized assignment problems. A deep Q-learning (DQL) approach is proposed in [17] to perform the user association between a terrestrial base station and a HAP drone based on the channel state information of the previous time slot. In addition to the above-mentioned UE's selection between terrestrial networks and non-terrestrial networks, there has been relevant research on the threeparty matching problem among users, HAP drones and satellites in remote areas without terrestrial network coverage.…”

Section: Related Workmentioning

confidence: 99%

Joint Drone Access and LEO Satellite Backhaul for a Space–Air–Ground Integrated Network: A Multi-Agent Deep Reinforcement Learning-Based Approach

Huang,

Xia,

Wang

et al. 2024

Drones

View full text Add to dashboard Cite

The space–air–ground integrated network can provide services to ground users in remote areas by utilizing high-altitude platform (HAP) drones to support stable user access and using low earth orbit (LEO) satellites to provide large-scale traffic backhaul. However, the rapid movement of LEO satellites requires dynamic maintenance of the matching relationship between LEO satellites and HAP drones. Additionally, different traffic types generated at HAP drones hold varying levels of values. Therefore, a tripartite matching problem among LEO satellites, HAP drones, and traffic types jointly considering multi-dimensional characteristics such as remaining visible time, channel condition, handover latency, and traffic storage capacity is formulated as mixed integer nonlinear programming to maximize the average transmitted traffic value. The traffic generation state for HAP drones is modeled as a mixture of stochasticity and determinism, which aligns with real-world scenarios, posing challenges for traditional optimization solvers. Thus, the original problem is decoupled into two independent sub-problems: traffic–drone matching and LEO–drone matching, which are addressed by mathematical simplification and multi-agent deep reinforcement learning with centralized training and decentralized execution, respectively. Simulation results verify the effectiveness and superiority of the proposed tripartite matching approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Joint Drone Access and LEO Satellite Backhaul for a Space–Air–Ground Integrated Network: A Multi-Agent Deep Reinforcement Learning-Based Approach

Huang,

Xia,

Wang

et al. 2024

Drones

View full text Add to dashboard Cite

show abstract

“…In [35], Khoshkbari et al proposed a novel deep Q-learning approach in which a satellite serves as an operative agent. This agent is responsible for scheduling each user to a terrestrial base station (TBS) or a high-altitude platform station (HAPS) within each time slot, utilizing channel state information (CSI) obtained from the preceding time slot.…”

Section: Related Workmentioning

confidence: 99%

DQN-GNN-Based User Association Approach for Wireless Networks

Alablani,

Alenazi

2023

Mathematics

View full text Add to dashboard Cite

In the realm of advanced mobile networks, such as the fifth generation (5G) and beyond, the increasing complexity and proliferation of devices and unique applications present a substantial challenge for User Association (UA) in wireless systems. The problem of UA in wireless networks is multifaceted and requires comprehensive exploration. This paper presents a pioneering approach to the issue, integrating a Deep Q-Network (DQN) with a Graph Neural Network (GNN) to enhance user-base station association in wireless networks. This novel approach surpasses recent methodologies, including Q-learning and max average techniques, in terms of average rewards, returns, and success rate. This superiority is attributed to its capacity to encapsulate intricate relationships and spatial dependencies among users and base stations in wireless systems. The proposed methodology achieves a success rate of 95.2%, outperforming other methodologies by a margin of up to 5.9%.

show abstract

“…The authors in [5] proposed an interference coordination method in integrated HAPS-terrestrial networks to maximize the sum data rate of the UEs. In [6], the authors designed a user association scheme based on the deep Q-learning (DQL) approach to maximize the network's sum data rate. In [7], we employed the NW-MMF objective function to design joint subcarrier and power allocation in a vHetNet.…”

Section: Introductionmentioning

confidence: 99%

Handling Interference in Integrated HAPS-Terrestrial Networks Through Radio Resource Management

Alidadi

Yadav

Abbasi

et al. 2022

IEEE Wireless Commun. Lett.

View full text Add to dashboard Cite

Located in the stratospheric layer of Earth's atmosphere, the high altitude platform station (HAPS) is a promising network infrastructure, which can bring significant advantages to sixth-generation (6G) and beyond wireless communications systems by forming vertical heterogeneous networks (vHetNets). However, if not dealt with properly, integrated networks suffer from a number of performance challenges compared to standalone networks. In harmonized integrated networks, where different tiers share the same frequency spectrum, interference is an important challenge to be addressed. This work focuses on an integrated HAPSterrestrial network, serving users in an overlapped urban geographic area, and formulates a fairness optimization problem, aiming to maximize the minimum spectral efficiency (SE) of the network. Due to the highly nonconvex nature of the formulated problem, we develop a fast converging iterative algorithm that designs the massive multiple-input multiple-output (mMIMO) beamforming weights and the user association scheme such that the propagated inter-and intra-tier interference is managed. Simulation results demonstrate the proposed algorithm's superiority over standalone terrestrial networks and baseline algorithms.

show abstract

User Association in a VHetNet With Delayed CSI: A Deep Reinforcement Learning Approach

Cited by 11 publications

References 19 publications

Joint Drone Access and LEO Satellite Backhaul for a Space–Air–Ground Integrated Network: A Multi-Agent Deep Reinforcement Learning-Based Approach

Joint Drone Access and LEO Satellite Backhaul for a Space–Air–Ground Integrated Network: A Multi-Agent Deep Reinforcement Learning-Based Approach

DQN-GNN-Based User Association Approach for Wireless Networks

Handling Interference in Integrated HAPS-Terrestrial Networks Through Radio Resource Management

Contact Info

Product

Resources

About