Multi-Agent Dynamic Area Coverage Based on Reinforcement Learning with Connected Agents

Abstract

Dynamic area coverage with small unmanned aerial vehicle (UAV) systems is one of the major research topics due to limited payloads and the difficulty of decentralized decision-making process. Collaborative behavior of a group of UAVs in an unknown environment is another hard problem to be solved. In this paper, we propose a method for decentralized execution of multi-UAVs for dynamic area coverage problems. The proposed decentralized decision-making dynamic area coverage (DDMDAC) method utilizes reinforcement learning (RL) where each UAV is represented by an intelligent agent that learns policies to create collaborative behaviors in partially observable environment. Intelligent agents increase their global observations by gathering information about the environment by connecting with other agents. The connectivity provides a consensus for the decision-making process, while each agent takes decisions. At each step, agents acquire all reachable agents' states, determine the optimum location for maximal area coverage and receive reward using the covered rate on the target area, respectively. The method was tested in a multi-agent actor-critic simulation platform. In the study, it has been considered that each UAV has a certain communication distance as in real applications. The results show that UAVs with limited communication distance can act jointly in the target area and can successfully cover the area without guidance from the central command unit.

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

“…PPO is the base DRL algorithm in this paper. Aydemir and Cetin [ 159 ] proposed a distributed system for the multi-UAV coverage in partially observable environments using DRL. Only the nearby robots share their state information and observations with each other.…”

Algebraic-Connectivity-Based Multi-USV Distributed Formation Method via Adding a Reverse Edge

“…Therefore, a relatively small algebraic connectivity is needed when forming a formation for the follower USVs. In addition, algebraic connectivity can also reflect the convergence rate of the collaborative multi-USV system [27,28]. When the formation of follower USVs guarantees relative dispersion and safety, it is best to have the fastest possible convergence rate.…”

Dağıtık pekiştirmeli öğrenme tabanlı çoklu insansız hava aracı ile ilgi çekici nokta kapsama

“…İHA'ların dağıtık olarak çalışmasına izin verilen bu yöntemde, hedef alanda İHA'lar arasındaki bağlantının kurulması da amaçlanmıştır. Aydemir ve Çetin [25], çoklu İHA kullanarak dinamik ortamda alan kapsamayı en yükseğe çıkarmaya çalışmıştır. Derin pekiştirmeli öğrenme ile modellenmiş ajanlar merkezi bir modül ile öğrenme gerçekleştirir; ancak yürütme aşamasında merkezi modül devreden çıkarılır.…”