Toward Autonomous Multi-UAV Wireless Network: A Survey of Reinforcement Learning-Based Approaches

Abstract

Unmanned aerial vehicle (UAV)-based wireless networks have received increasing research interest in recent years and are gradually being utilized in various aspects of our society. The growing complexity of UAV applications such as disaster management, plant protection, and environment monitoring, has resulted in escalating and stringent requirements for UAV networks that a single UAV cannot fulfill. To address this, multi-UAV wireless networks (MUWNs) have emerged, offering enhanced resource-carrying capacity and enabling collaborative mission completion by multiple UAVs. However, the effective operation of MUWNs necessitates a higher level of autonomy and intelligence, particularly in decision-making and multiobjective optimization under diverse environmental conditions. Reinforcement Learning (RL), an intelligent and goal-oriented decision-making approach, has emerged as a promising solution for addressing the intricate tasks associated with MUWNs. As one may notice, the literature still lacks a comprehensive survey of recent advancements in RL-based MUWNs. Thus, this paper aims to bridge this gap by providing a comprehensive review of RL-based approaches in the context of autonomous MUWNs. We present an informative overview of RL and demonstrate its application within the framework of MUWNs. Specifically, we summarize various applications of RL in MUWNs, including data access, sensing and collection, resource allocation for wireless connectivity, UAV-assisted mobile edge computing, localization, trajectory planning, and network security. Furthermore, we identify and discuss several open challenges based on the insights gained from our review.

Bayesian detection with feedback for cooperative spectrum sensing in cognitive UAV networks

“…Although there is a tradeoff between the sensing process time and utilization channel time, 30 these efforts to improve the efficiency of UAV CSS using SPRT are a new direction. In addition, due to the development of manufacturing as well artificial intelligence techniques, 31 machine learning empowerment has gradually become a new hot topic in UAV research, 32 but the learning process of a large number of training sequences also brings new problems to the spectrum decision-making of UAV CSS, such as high computational complexity and complexity, high latency, etc.…”

Trajectory-Aware Offloading Decision in UAV-Aided Edge Computing: A Comprehensive Survey

“…This approach complements terrestrial networks, particularly when the ground base station (GBS) servers are overloaded or unavailable. Owing to their high mobility and exceptional maneuverability, UAVs have been employed as MEC platforms to extend communication coverage and upgrade deployment efficiency [ 12 ]. Thus, UAV-aided MEC provides intensive flexibility and can convey better support for on-demand computing services than conventional MEC that relies on GBSs [ 13 ].…”

Joint Phase Shift Design and Resource Management for a Non-Orthogonal Multiple Access-Enhanced Internet of Vehicle Assisted by an Intelligent Reflecting Surface-Equipped Unmanned Aerial Vehicle

“…The reason for this is that 5G/B5Genabled IoV adopts a quasi-optic millimeter wave (mmWave) and visible light for communications [11][12][13]. In this situation, although the UAVs can facilitate line-of-sight (LoS) transmissions, the higher path loss attenuates the propagation signals between the UAV and cell edge vehicles (CEVs) and between the UAV and base station (BS) [14]. First, the higher path loss inherent in mmWave and visible-light communication systems presents a formidable obstacle for UAVs operating as relay nodes.…”