Trajectory Planning for UAV-Assisted Data Collection in IoT Network: A Double Deep Q Network Approach

Abstract

Unmanned aerial vehicles (UAVs) are becoming increasingly valuable as a new type of mobile communication device and autonomous decision-making device in many application areas, including the Internet of Things (IoT). UAVs have advantages over other stationary devices in terms of high flexibility. However, a UAV, as a mobile device, still faces some challenges in optimizing its trajectory for data collection. Firstly, the high complexity of the movement action and state space of the UAV’s 3D trajectory is not negligible. Secondly, in unknown urban environments, a UAV must avoid obstacles accurately in order to ensure a safe flight. Furthermore, without a priori wireless channel characterization and ground device locations, a UAV must reliably and safely complete the data collection from the ground devices under the threat of unknown interference. All of these require the proposing of intelligent and automatic onboard trajectory optimization techniques. This paper transforms the trajectory optimization problem into a Markov decision process (MDP), and deep reinforcement learning (DRL) is applied to the data collection scenario. Specifically, the double deep Q-network (DDQN) algorithm is designed to address intelligent UAV trajectory planning that enables energy-efficient and safe data collection. Compared with the traditional algorithm, the DDQN algorithm is much better than the traditional Q-Learning algorithm, and the training time of the network is shorter than that of the deep Q-network (DQN) algorithm.

Adaptive Multi-Surface Sliding Mode Control with Radial Basis Function Neural Networks and Reinforcement Learning for Multirotor Slung Load Systems

“…Some work implementing DQN agents for the control of inverted pendula presently exists [26][27][28], although no existing work has been carried out for slung load systems, or for multilink pendula. Some research has been carried out with UAVs [29,30], although as is the case with RL, this does not include slung load transport.…”

Deep Reinforcement Learning-Based 3D Trajectory Planning for Cellular Connected UAV

“…where θ denotes neural network parameter vector. However, the direct utilization of (31) in the standard training algorithm may give rise to the issue of overestimating Q value, thereby leading to learning instability and inefficiency. To address this challenge, we introduce Double DQN into our research, aiming to mitigate overestimation.…”

“…The work referenced in [29] introduces a two-step centralized development system for 3D path planning of drone swarms. Additionally, Both articles [30,31] take into account energy consumption during the 3D trajectory planning process for UAVs. Ref.…”