Task Assignment of UAV Swarms Based on Deep Reinforcement Learning

Liu, Bo; Wang, Shulei; Li, Qinghua; Zhao, Xinyang; Pan, Yunqing; Wang, Changhong

doi:10.3390/drones7050297

Cited by 9 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the above formula, a k represents the important neighborhood agent for agent j. Then the Q-value of each agent is shown in Equation (17). Note that the Q-value here is a partially observable Q-value.…”

Section: Mean Field Modulementioning

confidence: 99%

“…Shi et al [3] successfully migrated MARL to real-world multi-UAV handling tasks by employing Recurrent Multi-Agent Deep Deterministic Policy Gradient (R-MADDPG) and domain randomization techniques. Liu et al [17] proposed the Extensible Multi-Agent Deep Deterministic Policy Gradient (Ex-MADDPG) algorithm to address dynamic task assignment problems in UAV swarms, in which the practicality and effectiveness of the Ex-MADDPG algorithm were validated using a swarm of nine UAVs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph Attention Network for UAV Swarms

Yang

Liu

Zhou

et al. 2023

Drones

View full text Add to dashboard Cite

Multiple unmanned aerial vehicles (Multi-UAV) systems have recently demonstrated significant advantages in some real-world scenarios, but the limited communication range of UAVs poses great challenges to multi-UAV collaborative decision-making. By constructing the multi-UAV cooperation problem as a multi-agent system (MAS), the cooperative decision-making among UAVs can be realized by using multi-agent reinforcement learning (MARL). Following this paradigm, this work focuses on developing partially observable MARL models that capture important information from local observations in order to select effective actions. Previous related studies employ either probability distributions or weighted mean field to update the average actions of neighborhood agents. However, they do not fully consider the feature information of surrounding neighbors, resulting in a local optimum often. In this paper, we propose a novel partially multi-agent reinforcement learning algorithm to remedy this flaw, which is based on graph attention network and partially observable mean field and is named as the GPMF algorithm for short. GPMF uses a graph attention module and a mean field module to describe how an agent is influenced by the actions of other agents at each time step. The graph attention module consists of a graph attention encoder and a differentiable attention mechanism, outputting a dynamic graph to represent the effectiveness of neighborhood agents against central agents. The mean field module approximates the effect of a neighborhood agent on a central agent as the average effect of effective neighborhood agents. Aiming at the typical task scenario of large-scale multi-UAV cooperative roundup, the proposed algorithm is evaluated based on the MAgent framework. Experimental results show that GPMF outperforms baselines including state-of-the-art partially observable mean field reinforcement learning algorithms, providing technical support for large-scale multi-UAV coordination and confrontation tasks in communication-constrained environments.

show abstract

Section: Mean Field Modulementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph Attention Network for UAV Swarms

Yang

Liu

Zhou

et al. 2023

Drones

View full text Add to dashboard Cite

show abstract

“…This approach effectively balances the conflicting objectives of safety and performance within the affine formation in obstacle‐rich environments. By employing this method, agents can navigate through complex environments while maintaining the desired formation, ensuring a harmonious interplay between safety and efficiency. (3)The efficiency and feasibility of our proposed method are validated through simulations and real‐world experiments with a fleet of self‐developed unmanned aerial vehicles called scit‐mini [32]. A video demonstrating the real‐world experiment can be viewed online at https://youtu.be/5eBN9m22Nww.…”

Section: Introductionmentioning

confidence: 99%

Safe affine formation using terminal sliding mode control with input constraints

Liu,

Wang,

Wang

et al. 2024

IET Control Theory & Appl

Self Cite

View full text Add to dashboard Cite

Formation control is a fundamental task in the realm of autonomous multiagent systems. To drive a group of agents to maneuver continuously with the desired formation, this paper studies the finite‐time affine formation control problem with disturbances, input constraints and safety guarantee. A non‐singular terminal sliding mode control (NTSMC) is implemented to achieve finite‐time convergence of all followers to their desired positions. Additionally, an auxiliary system is deployed to address input constraints resulting from the physical properties of the affine formation system. To mitigate the impact of lumped disturbances, a finite‐time disturbance observer (FTDO) is employed to estimate the disturbances and compensate for their effects. Based on FTDO, the auxiliary system and the above NTSMC, a finite‐time robust controller is developed as the nominal controller. By modifying the nominal controllers to comply with safety constraints, control barrier functions are employed to ensure the safety of the formation system in obstacle‐filled environment. Finally, the effectiveness and feasibility of this method are validated through simulations and real‐world experiments.

show abstract

“…Static formations of drones are extensively used for various applications 1 , such as those involving continuous aerial observations 2 – 4 , the assignment of different tasks 5 , intruder monitoring 6 , air quality measurements 7 , 8 or the interception of rockets and ballistic missiles 9 . The formation should be maintained even in the presence of unwanted or undesired intruders that introduce a disturbance to the formation.…”

Section: Introductionmentioning

confidence: 99%

Multivariate linear regression model based on cross-entropy for estimating disorganisation in drone formations

Gackowska,

Cofta,

Śrutek

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Static formations of swarms of rotorcraft drones, used for example in disaster management, are subject to intrusions, and must bear the cost of holding the formation while avoiding collisions which leads to their increased energy consumption. While the behaviour of the intruder is unpredictable, the formation can have its parameters set to try to balance the cost of avoidance with its functionality. The novel model presented in this paper assists in the selection of parameter values. It is based on multivariate linear regression, and provides an estimate of the average disturbance caused by an intruder as a function of the values of the parameters of a formation. Cross-entropy is used as a metric for the disturbance, and the data based are generated through simulations. The model explains up to 54.4% of the variability in the value of the cross-entropy, providing results that are twice as good as the baseline estimator of the mean cross-entropy.

show abstract

Task Assignment of UAV Swarms Based on Deep Reinforcement Learning

Cited by 9 publications

References 30 publications

Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph Attention Network for UAV Swarms

Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph Attention Network for UAV Swarms

Safe affine formation using terminal sliding mode control with input constraints

Multivariate linear regression model based on cross-entropy for estimating disorganisation in drone formations

Contact Info

Product

Resources

About