UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Abstract

Focusing on the problem of multi-UAV cooperative air combat decision-making, a multi-UAV cooperative maneuvering decision-making approach is proposed based on multi-agent deep reinforcement learning (MARL) theory. First, the multi-UAV cooperative short-range air combat environment is established. Then, by combining the value-decomposition networks (VDNs) deep reinforcement learning theory with the embedded expert collaborative air combat experience reward function, an air combat cooperative strategy framework is proposed based on the networked decentralized partially observable Markov decision process (NDec-POMDP). The air combat maneuvering strategy is then optimized to improve the cooperative degree between UAVs in cooperative combat scenarios. Finally, multi-UAV cooperative air combat simulations are carried out and the results show the feasibility and effectiveness of the proposed cooperative air combat decision-making framework and method.

A Policy Gradient Algorithm to Alleviate the Multi-Agent Value Overestimation Problem in Complex Environments

“…Popular multi-agent reinforcement learning algorithms integrate value-learning and strategy-learning structures. It is worth noting that reference [14] uses the multi-agent depth strategy gradient algorithm to solve the challenges brought by the complex and uncertain dynamic environment encountered in the UAV cluster confrontation process. Reference [15] proposes a multi-agent cooperative combat simulation algorithm based on reinforcement learning to achieve a balanced decision-making method.…”

Distributed Momentum-based Frank-Wolfe Algorithm for Stochastic Optimization

“…where X ⊂ R p is a convex and compact feasible set. Problems of the form (1) lie at the heart of machine learning and adaptive filtering, emerging in e.g., clustering, classification, energy management, and resource allocation [8]- [11].…”