DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET

Abstract

Unmanned aerial vehicles (UAVs) are important equipment for efficiently executing search and rescue missions in disaster or air-crash scenarios. Each node can communicate with the others by a routing protocol in UAV ad hoc networks (UANETs). However, UAV routing protocols are faced with the challenges of high mobility and limited node energy, which hugely lead to unstable link and sparse network topology due to premature node death. Eventually, this severely affects network performance. In order to solve these problems, we proposed the deep-reinforcement-learning-based geographical routing protocol of considering link stability and energy prediction (DSEGR) for UANETs. First of all, we came up with the link stability evaluation indicator and utilized the autoregressive integrated moving average (ARIMA) model to predict the residual energy of neighbor nodes. Then, the packet forward process was modeled as a Markov Decision Process, and according to a deep double Q network with prioritized experience replay to learn the routing-decision process. Meanwhile, a reward function was designed to obtain a better convergence rate, and the analytic hierarchy process (AHP) was used to analyze the weights of the considered factors in the reward function. Finally, to verify the effectiveness of DSEGR, we conducted simulation experiments to analyze network performance. The simulation results demonstrate that our proposed routing protocol remarkably outperforms others in packet delivery ratio and has a faster convergence rate.

Routing with Multi-Criteria QoS for Flying Ad-hoc Networks (FANETs)

“…Simulation was conducted for configuration parameters given in Table I. Comparison of proposed solution's performance is done against Q-learning based QoS routing proposed by Xinchen et al [ 24], link stability-based routing proposed by Zhang et al [21], trajectory aided OLSR routing proposed by Hou et al [7].…”

“…Overhead has reduced in proposed solution, due to use of less maintenance overhead. Zhang et al [21] Xinchen et al [24] www.ijacsa.thesai.org V. DISCUSSION AND RESULTS…”

“…Life time maximization with joint consideration for QoS is in scope of future work. Zhang et al [21] Xinchen et al [24]…”

“…The maintenance overhead is high in this approach. Zhang et al [21] proposed a deep-reinforcement-learningbased geographical routing protocol based on link stability and energy prediction. The next hop selection decision was done using reinforcement learning.…”

Multi-Objective Message Routing in Electric and Flying Vehicles Using a Genetics Algorithm

“…Nevertheless, several existing solutions have tried utilizing SDN or a multi-objective routing for ground vehicles [ 10 , 11 , 12 ]. Moreover, routing in UAVs which considers metrics such as charging stations, VANET supporting networks, and trajectory-based systems has also been proposed [ 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. An exhaustive review of existing schemes identified that a routing scheme should not consider a single objective while disregarding its byproduct’s impact.…”