Autonomous Decision-Making Method for Combat Mission of UAV based on Deep Reinforcement Learning

Xu, Junqi; Guo, Qing; Xiao, Lei; Li, Zhaoyi; Zhang, Gaowei

doi:10.1109/iaeac47372.2019.8998066

Cited by 31 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We defined two WEZ rewards: R shoot (range, AT A) is the effective shooting reward, and R beHit (range, AA) is a penalty when the target shoots the ownship. The agent gets rewards for every step according to equations ( 7) and (8).…”

Section: ) Wezmentioning

confidence: 99%

Deep Reinforcement Learning-Based Air-to-Air Combat Maneuver Generation in a Realistic Environment

et al. 2023

View full text Add to dashboard Cite

Artificial intelligence is becoming increasingly important in the air combat domain. Most air combat research now assumes that all aircraft information is known. In practical applications, however, some aircraft information, such as their position, attitude, velocity, etc., can be incorrect or impossible to obtain due to realistic limitations and sensor errors. In this paper, we propose a deep reinforcement learningbased framework for developing a model capable of performing within visual range (WVR) air-to-air combat under the conditions of a partially observable Markov decision process (POMDP) with insufficient information. To deal robustly with such a situation, we use recurrent neural networks and apply a soft actorcritic (SAC) algorithm to cope effectively with realistic limitations and sensor errors. Additionally, to raise the efficiency and effectiveness of learning, we apply the curriculum learning technique to restrict the scope of exploration in state space. Finally, simulations and experiments show that the proposed techniques can deal with practical problems caused by sensor limitations and errors in a noisy environment while also being efficient and effective in reducing the training time for learning.INDEX TERMS Air-to-air combat, limitation and error of sensors, recurrent neural network, reinforcement learning, soft actor-critic.

show abstract

Section: ) Wezmentioning

confidence: 99%

Deep Reinforcement Learning-Based Air-to-Air Combat Maneuver Generation in a Realistic Environment

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In [32], a double-screening sampling method is designed, which combines deep learning with deep deterministic strategy gradient algorithm to break the correlation of continuous experiments in the experience base and improve the convergence of the algorithm. In [33], based on the complexity and dynamics of the future battlefield, an autonomous decision-making method for UAV is developed by combining the deep belief network decision-making model with genetic algorithm. In UAV collaborative task assignment, most of the improved algorithms based on reinforcement learning are based on the fact that Q-table stored data in reinforcement learning is not easy to be too large and using neural networks to transform it into deep reinforcement learning, but for discrete data processing like task assignment, deep learning is time-consuming and does not take advantage in fast task assignment.…”

Section: Literature Reviewmentioning

confidence: 99%

Multi-UAV Cooperative Task Assignment Based on Half Random Q-Learning

Zhu

Fang

2021

Symmetry

View full text Add to dashboard Cite

Unmanned aerial vehicle (UAV) clusters usually face problems such as complex environments, heterogeneous combat subjects, and realistic interference factors in the course of mission assignment. In order to reduce resource consumption and improve the task execution rate, it is very important to develop a reasonable allocation plan for the tasks. Therefore, this paper constructs a heterogeneous UAV multitask assignment model based on several realistic constraints and proposes an improved half-random Q-learning (HR Q-learning) algorithm. The algorithm is based on the Q-learning algorithm under reinforcement learning, and by changing the way the Q-learning algorithm selects the next action in the process of random exploration, the probability of obtaining an invalid action in the random case is reduced, and the exploration efficiency is improved, thus increasing the possibility of obtaining a better assignment scheme, this also ensures symmetry and synergy in the distribution process of the drones. Simulation experiments show that compared with Q-learning algorithm and other heuristic algorithms, HR Q-learning algorithm can improve the performance of task execution, including the ability to improve the rationality of task assignment, increasing the value of gains by 12.12%, this is equivalent to an average of one drone per mission saved, and higher success rate of task execution. This improvement provides a meaningful attempt for UAV task assignment.

show abstract

“…Reference [7] proposed a multi-UAV distributed intelligent self-organization algorithm, which decomposes the optimization problem of the cluster reconnaissanceattack task into multiple local optimization problems, and realizes global optimization decision-making through information exchange between the cluster and the environment and within the cluster. Reference [8] used a deep learning method to construct a task decision-making model for typical cluster tasks such as area reconnaissance, and then optimized the decision-making model based on a genetic algorithm, providing effective support for offline learning and online decision-making of the cluster. However, existing research on UAV cluster autonomous decision-making problems is relatively scarce from a multi-task perspective.…”

Section: Introudctionmentioning

confidence: 99%

Research on Unmanned Aerial Vehicle Cluster Collaborative Countermeasures Based on Dynamic Non-Zero-Sum Game under Asymmetric and Uncertain Information

Wang

Liang

et al. 2023

Aerospace

View full text Add to dashboard Cite

Unmanned aerial vehicle (UAV) swarm coordinated confrontation is a hot topic in academic research at home and abroad, and dynamic maneuver decision-making is one of the most important research fields for UAV countermeasures. Aiming at the complexity, uncertainty and confrontation of UAV cooperative confrontation, concepts such as relative advantage degree and advantage coefficient are introduced, and game theory is used as a framework to construct a dynamic non-zero-sum game UAV cluster cooperative confrontation decision-making model, and finally convert it into an optimization problem. On this basis, using the Nash equilibrium solution method of multi-strategy fusion particle swarm algorithm, by introducing adaptive inertia weight and local mutation strategy, while enhancing the diversity of the population, it can ensure the local accurate search ability of the particle swarm. The simulation results of the example are verified. The effectiveness of the proposed model and method is confirmed.

show abstract

Autonomous Decision-Making Method for Combat Mission of UAV based on Deep Reinforcement Learning

Cited by 31 publications

References 9 publications

Deep Reinforcement Learning-Based Air-to-Air Combat Maneuver Generation in a Realistic Environment

Deep Reinforcement Learning-Based Air-to-Air Combat Maneuver Generation in a Realistic Environment

Multi-UAV Cooperative Task Assignment Based on Half Random Q-Learning

Research on Unmanned Aerial Vehicle Cluster Collaborative Countermeasures Based on Dynamic Non-Zero-Sum Game under Asymmetric and Uncertain Information

Contact Info

Product

Resources

About