Distributed reinforcement learning for flexible UAV swarm control with transfer learning capabilities

Abstract

Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs) in monitoring and remote area surveillance applications has become widespread thanks to the price reduction and the increased capabilities of drones. The drones in the swarm need to cooperatively explore an unknown area, in order to identify and monitor interesting targets, while minimizing their movements. In this work, we propose a distributed Reinforcement Learning (RL) approach that scales to larger swarms without modifications. The proposed framework relies on the possibility for the UAVs to exchange some information through a communication channel, in order to achieve context-awareness and implicitly coordinate the swarm's actions. Our experiments show that the proposed method can yield effective strategies, which are robust to communication channel impairments, and that can easily deal with nonuniform distributions of targets and obstacles. Moreover, when agents are trained in a specific scenario, they can adapt to a new one with minimal additional training. We also show that our approach achieves better performance compared to a computationally intensive look-ahead heuristic.

Multi-UAV Path Planning for Wireless Data Harvesting With Deep Reinforcement Learning

“…Other MARL path planning approaches to minimize the age of information of collected data include [12] and [13]. In [24], a swarm of UAVs on a target detection and tracking mission in an unknown environment is controlled through a distributed DQN approach. While the authors also use convolutional processing to feed map information to the agents, the map is initially unknown and has to be explored to detect the targets.…”

An Integrated Approach for Post-Disaster Flood Management Via the Use of Cutting-Edge Technologies and UAVs: A Review

“…The UAV route can be optimised to provide maximum area coverage of the area in minimum time and cost [83] (Figure 6). The UAV swarm will collect the images of the region and provide the gathered data to the control centre [85][86][87][88]. Recently, Albani et al [85] applied a macroscopic model for monitoring an area using UAVs.…”

“…The UAV swarm will collect the images of the region and provide the gathered data to the control centre [85][86][87][88]. Recently, Albani et al [85] applied a macroscopic model for monitoring an area using UAVs. Parametriasition was proposed for efficient allocation of the UAVs; abstract multiple-agent simulations were conducted to deploy UAVs in multiple areas, and simulation of UAV swarm was carried out for mapping the areas.…”

Machine Learning Methods for UAV Flocks Management-A Survey

“…In the work of Venturini et al [115], the authors considered a general MARL framework for the initial exploration and surveillance of a swarm of independent UAVs. Their scheme followed the framework in which observations of other agents are used to make decisions and to avoid collision, thereby encouraging cooperation.…”

“…Challenge Optimization ML Criteria Method Olfati-Saber [81] Flocking challenges connectivity, energy Distributed flocking algorithms Maza et al [76], [77] Architecture execution cost Temporal planning, contract net Quintero [92] Localization distance and heading DP Xu et al [122] Ensuring flocking rules cohesion, separation, NDP and alignment Hung and Givigi [49] Coordination flocking cost function Q-learning Tsai [113] Vision-based collision HTER RNN avoidance Jafrai et al [52] Flocking design multi-objective properties Bio-inspired RL Venturini et al [115] Exploration and target reaching efficiency Deep Q-learning surveillance Anicho et al [6] Coordinating coverage RL and SI Sharma and Ghose [104] Collision avoidance swarm size and stability Swarm laws Decentralized alg. ABSs with a limited coverage range.…”