Deep Reinforcement Learning Based Energy Efficient Multi-UAV Data Collection for IoT Networks

Abstract

Unmanned aerial vehicles (UAVs) are regarded as an emerging technology, which can be effectively utilized to perform the data collection tasks in the Internet of Things (IoT) networks. However, both the UAVs and the sensors in these networks are energylimited devices, which necessitates an energy efficient data collection procedure to ensure the network lifetime. In this paper, we propose a multi-UAV-assisted network, where the UAVs fly to the ground sensors and control the sensor's transmit power during the data collection time. Our goal is to minimize the total energy consumption of the UAVs and the sensors, which is needed to accomplish the data collection mission. We formulate this problem into three sub-problems of single UAV navigation, sensor power control as well as multi-UAV scheduling and model each part as a finite-horizon Markov Decision Process (MDP). We deploy deep reinforcement learning (DRL)-based frameworks to solve each part. Specifically, we use deep deterministic policy gradient (DDPG) method to generate the best trajectory for the UAVs in an obstacle-constraint environment, given its starting position and the target sensor. We also deploy DDPG to control the sensor's transmit power during data collection. To schedule activity plans for each UAV to visit the sensors, we propose a multi-agent deep Q-learning (DQL) approach by taking the total energy consumption of the UAVs on each path into account. Our simulations show that the UAVs can find a safe and optimal path for each of their trips. Continuous power control of the sensors achieves better performance over the fixed power approaches in terms of the total energy consumption during data collection. In addition, compared to the two commonly used baselines, our scheduling framework achieves better and near-optimal results.

Multi‐UAV path planning utilizing the PGA algorithm for terrestrial IoT sensor network under ISAC framework

“…Moreover Reference 20, explored an energy‐minimizing and clustering approach for sequential trajectory planning, showcasing the effectiveness of DRL in single UAV operations. The authors of 21 put forth a DRL‐based platform for data gathering with multiple UAVs. In Reference 22, energy limitations and sensing information update scenarios for data gathering with multiple UAVs were considered, employing a deep Q‐network and showcasing promising performance.…”

Reinforcement Learning-based Topology-Aware Routing Protocol with Priority Scheduling for Internet of Drones in Agriculture Application

“…Using the UAV's starting spot and the destination device, [15] employed the deep deterministic policy gradient (DDPG) approach that create the optimum design for the UAV inside an extra hurdle context. In [16] the author made suitable for low resource usage, a pervasive farming mobile sensor network-based threshold built-in MAC routing protocol (TBMP) has been developed.…”

“…The performance of our proposed reinforcement learning-based topology-aware routing protocol with priority scheduling (RLTARP) is carried out by compared with three existing methods such as DroneCOCoNet [12], Markov decision process (MDP) [15] and deep deterministic policy gradient (DDPG) [16].…”

On collaborative multi-UAV trajectory planning for data collection

“…However, it also experiences small-scale fading caused by the presence of rich scattering in the environment [32]. For the up-link channel between UAV and IoT node, we use the rician fading channel [33]. The channel model between UAV j ∈ U and IoT node i ∈ I at time t ∈ T can be expressed as:…”

“…where d j,i denotes the horizontal distance between UAV j and IoT node i at height H. β 0 is the average channel power gain at the reference distance of d 0 = 1 m and α is the path loss exponent that usually has a value between 2 and 6 [33].…”