Research on reinforcement learning-based safe decision-making methodology for multiple unmanned aerial vehicles

Yue, Longfei; Yang, Rennong; Zhang, Ying; Zuo, Jialiang

doi:10.3389/fnbot.2022.1105480

Cited by 4 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address a cooperative search problem, Zhang et al used an improved particle swarm optimization algorithm to allocate the UAV reconnaissance area and maximize the utility of the UAV cluster [23].Of course, reinforcement learning is also a good option. Yue et al proposed a secure transfer soft-AC algorithm with security constraints for maximizing revenue [24]. Baek adopted distributed algorithms so UAV swarms could track ground targets and proposed an optimal sensor management technology and consensus-based decision algorithm to minimize the uncertainty of the target location [25].…”

Section: A Related Workmentioning

confidence: 99%

A Sequential Decision Algorithm of Reinforcement Learning for Composite Action Space

Gao,

Wang,

Zhang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

It is the key research object of electronic warfare to use UAV( Unmanned Aerial Vehicle) clusters to carry out electronic countermeasure tasks. The UAV carries loads such as reconnaissance and interference at the same time, which makes it necessary to simultaneously decide multiple types of actions-namely, compound actions-which poses a challenge to intelligent decision-making algorithms.Considering the problem of action-space dimensional complexity and weak collaboration between decisions in multi-agent scenarios with composite actions, this study proposed a decision algorithm involving a multiagent reinforcement-learning sequence, which combined joint composite actions into sequential decision, reducing the difficulty of a single decision and enhancing the collaboration between various agents and their individual decisions. Because long decision sequences required better depth modeling and had high variance, a DeLighT module was added to the naïve transformer model to increase the depth and baseline techniques, which were used to reduce the variance in the value estimation. The simulated results verified the effectiveness of the proposed algorithm in the UAV cooperative combat scenario, where each agent had a composite action space and showed better performance than the existing algorithms.

show abstract

Section: A Related Workmentioning

confidence: 99%

A Sequential Decision Algorithm of Reinforcement Learning for Composite Action Space

Gao,

Wang,

Zhang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Moreover, deep reinforcement learning (DRL) (Mnih et al, 2015 ) combines DL and RL to implement end-to-end learning. It makes RL no longer limited to low-dimensional space and greatly expands the scope of application of RL (Wang C. et al, 2020 ; Chane-Sane et al, 2021 ; He L. et al, 2021 ; Kiran et al, 2021 ; Luo et al, 2021 ; Wu et al, 2021 ; Yan et al, 2022 ; Yue et al, 2023 ; Zhao et al, 2023 ). Wu et al ( 2021 ) introduced a curiosity-driven method into DRL to improve training efficiency and performance in autonomous driving tasks.…”

Section: Introductionmentioning

confidence: 99%

Multi-UAV simultaneous target assignment and path planning based on deep reinforcement learning in dynamic multiple obstacles environments

Kong,

Zhou,

et al. 2024

Front. Neurorobot.

View full text Add to dashboard Cite

Target assignment and path planning are crucial for the cooperativity of multiple unmanned aerial vehicles (UAV) systems. However, it is a challenge considering the dynamics of environments and the partial observability of UAVs. In this article, the problem of multi-UAV target assignment and path planning is formulated as a partially observable Markov decision process (POMDP), and a novel deep reinforcement learning (DRL)-based algorithm is proposed to address it. Specifically, a target assignment network is introduced into the twin-delayed deep deterministic policy gradient (TD3) algorithm to solve the target assignment problem and path planning problem simultaneously. The target assignment network executes target assignment for each step of UAVs, while the TD3 guides UAVs to plan paths for this step based on the assignment result and provides training labels for the optimization of the target assignment network. Experimental results demonstrate that the proposed approach can ensure an optimal complete target allocation and achieve a collision-free path for each UAV in three-dimensional (3D) dynamic multiple-obstacle environments, and present a superior performance in target completion and a better adaptability to complex environments compared with existing methods.

show abstract

Exoatmospheric Evasion Guidance Law with Total Energy Limit via Constrained Reinforcement Learning

Yan,

Yang,

Zhao

et al. 2024

Int. J. Aeronaut. Space Sci.

View full text Add to dashboard Cite

Due to the lack of aerodynamic forces, the available propulsion for exoatmospheric pursuit-evasion problem is strictly limited, which has not been thoroughly investigated. This paper focuses on the evasion guidance in an exoatmospheric environment with total energy limit. A Constrained Reinforcement Learning (CRL) method is proposed to solve the problem. Firstly, the acceleration commands of the evader are defined as cost and an Actor-Critic-Cost (AC2) network structure is established to predict the accumulated cost of a trajectory. The learning objective of the agent becomes to maximize cumulative rewards while satisfying the cost constraint. Secondly, a Maximum-Minimum Entropy Learning (M2EL) method is proposed to minimize the randomness of acceleration commands while preserving the agent’s exploration capability. Our approaches address two challenges in the application of reinforcement learning: constraint specification and precise control. The well-trained agent is capable of generating accurate commands while satisfying the specified constraints. The simulation results indicate that the CRL and M2EL methods can effectively control the agent’s energy consumption within the specified constraints. The robustness of the agent under information error is also validated.

show abstract

Research on reinforcement learning-based safe decision-making methodology for multiple unmanned aerial vehicles

Cited by 4 publications

References 25 publications

A Sequential Decision Algorithm of Reinforcement Learning for Composite Action Space

A Sequential Decision Algorithm of Reinforcement Learning for Composite Action Space

Multi-UAV simultaneous target assignment and path planning based on deep reinforcement learning in dynamic multiple obstacles environments

Exoatmospheric Evasion Guidance Law with Total Energy Limit via Constrained Reinforcement Learning

Contact Info

Product

Resources

About