Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks

Abstract

A flying ad hoc network (FANETs), also known as a swarm of unmanned aerial vehicles (UAVs), can be deployed in a wide range of applications including surveillance, monitoring, and emergency communications. UAVs must perform real-time communication among themselves and the base station via an efficient routing protocol. However, designing an efficient multihop routing protocol for FANETs is challenging due to high mobility, dynamic topology, limited energy, and short transmission range. Recently, owing to the advantages of multi-objective optimization, Q-learning (QL)-based position-aware routing protocols have improved the performance of routing in FANETs. In his article, we provide a comprehensive review of existing QL-based position-aware routing protocols for FANETs. We rigorously address dynamic topology, mobility models, and the relationship between QL and routing in FANETs, and extensively review the existing QL-based position-aware routing protocols along with their advantages and limitations. Then, we compare the reviewed protocols qualitatively in terms of operational features, characteristics, and performance metrics. We also discuss important open issues and research challenges with potential research directions.

Drone Routing for Drone-Based Delivery Systems: A Review of Trajectory Planning, Charging, and Security

“…In multi-UAV systems, selecting the path with the most effective route is a challenging issue that must be solved to minimize the overall energy consumption and travel distance [ 33 ]. The joint consideration of the topology and routing to select the optimal path is shown to be successful in minimizing delay as well as energy consumption [ 64 , 65 ]. Thus, to meet the excessive parcel delivery requirements, multi-UAV deployment can be a potential solution [ 66 ].…”

Reinforcement-Based Clustering in Flying Ad-Hoc Networks for Serving Vertical and Horizontal Routing

“…The action represents the set of weight values used to calculate the overall utility that will make the node a cluster head. When using Q-learning or DQN or DDPG, we perform quantization using Equation (15).…”

“…Consequently, this increases the difficulty in maintaining a connected network and ensuring efficient data transmission. As a result, communication protocols for FANETs must be robust enough to handle the challenges posed by low-density networks [15].…”

“…Recently, RL has been widely employed in FANETs to build routing protocols for smooth collaborative trajectory planning for swarms of unmanned aerial vehicles. A model-free, value-based off-policy RL approach called Q-learning (QL) can quickly determine the best course of action based on prior knowledge of the environment, even without the help of a central controller [15]. However, designing an RL-based protocol in FANETs is not a straightforward process.…”

LSTDA: Link Stability and Transmission Delay Aware Routing Mechanism for Flying Ad-Hoc Network (FANET)

“…The proposed approach isolates and selects the best assignments for clustered components, and evaluation is performed using two clustering schemes: Bio-Inspired Clustering Schemes for FANET (BICSF) and Energy-Attention Link-Based Clustering (EALC) based on Health, High PDR, Low Latency (EMASS) Guarantees Clustering. In [19], a boosting method for the Optimized Link State Routing Protocol (OLSR) called the Whale Optimization Algorithm (WOA) is presented to stabilize energy productivity in FANET. The proposed approach is compared to the existing work at a high level.…”