Research on Air Confrontation Maneuver Decision-Making Method Based on Reinforcement Learning

Zhang, Xianbing; Liu, Guoqing; Yang, Chaojie; Wu, Jiang

doi:10.3390/electronics7110279

Cited by 59 publications

(33 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The third scenario is red agent fleeing. We compare our method with AC [28] and DDPG [29] in the three scenarios. During training, three methods use same environment models and reward models proposed in the paper.…”

Section: A Effectiveness Comparisonmentioning

confidence: 99%

“…The method of offline learning and online decisionmaking can satisfy both the time limit and the optimal solution requirement. The application of reinforcement learning in air combat is mainly based on the value function search [25][26][27] and policy search [28,29]. Literature [25] applies the Q-learning method to air combat, but does not perform separate learning for the red and blue agents.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Application of Deep Reinforcement Learning in Maneuver Planning of Beyond-Visual-Range Air Combat

Yang

Zuo

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Beyond-visual-range (BVR) engagement becomes more and more popular in the modern air battlefield. The key and difficulty for pilots in the fight is maneuver planning, which reflects the tactical decision-making capacity of the both sides and determinates success or failure. In this paper, we propose an intelligent maneuver planning method for BVR combat with using an improved deep Q network (DQN). First, a basic combat environment builds, which mainly includes flight motion model, relative motion model and missile attack model. Then, we create a maneuver decision framework for agent interaction with the environment. Basic perceptive variables are constructed for agents to form continuous state space. Also, considering the threat of each side missile and the constraint of airfield, the reward function is designed for agents to training. Later, we introduce a training algorithm and propose perceptional situation layers and value fitting layers to replace policy network in DQN. Based on long short-term memory (LSTM) cell, the perceptional situation layer can convert basic state to high-dimensional perception situation. The fitting layer does well in mapping action. Finally, three combat scenarios are designed for agent training and testing. Simulation result shows the agent can avoid the threat of enemy and gather own advantages to threat the target. It also proves the models and methods of agents are valid and intelligent air combat can be realized.

show abstract

Section: A Effectiveness Comparisonmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Application of Deep Reinforcement Learning in Maneuver Planning of Beyond-Visual-Range Air Combat

Yang

Zuo

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The demand for autonomous decision-making algorithms to support automated air confrontation systems is growing. The work by Zhang et al [12] addresses such demand by presenting the development of a super-horizon air confrontation training environment. The authors employ computational intelligence approaches, including reinforcement learning and neural networks, to create a self-learning air confrontation maneouver decision making system, which is tested by means of complex simulations of different air confrontation situations.…”

Section: The Present Special Issuementioning

confidence: 99%

Autonomous Control of Unmanned Aerial Vehicles

Becerra

2019

Electronics

View full text Add to dashboard Cite

show abstract

“…References [15,16] use the Bayesian theory to select the optimal discrete maneuver for maneuvering decision-making; it requires high data accuracy. References [17,18] use a deep reinforcement learning technique to deal with maneuvering decision-making problems, with low realtime performance.…”

Section: Introductionmentioning

confidence: 99%

Moving Time UCAV Maneuver Decision Based on the Dynamic Relational Weight Algorithm and Trajectory Prediction

Xie

Ding

Wei

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

To improve the accuracy and real-time performance of autonomous decision-making by the unmanned combat aerial vehicle (UCAV), a decision-making method combining the dynamic relational weight algorithm and moving time strategy is proposed, and trajectory prediction is added to maneuver decision-making. Considering the lack of continuity and diversity of air combat situation reflected by the constant weight in situation assessment, a dynamic relational weight algorithm is proposed to establish an air combat situation system and adjust the weight according to the current situation. Based on the dominance function, this method calculates the correlation degree of each subsituation and the total situation. According to the priority principle and information entropy theory, the hierarchical fitting function is proposed, the association expectation is calculated by using if-then rules, and the weight is dynamically adjusted. In trajectory prediction, the online sliding input module is introduced, and the long- and short-term memory (LSTM) network is used for real-time prediction. To further improve the prediction accuracy, the adaptive boosting (Ada) method is used to build the outer frame and compare with three traditional prediction networks. The results show that the prediction accuracy of Ada-LSTM is better. In the decision-making method, the moving time optimization strategy is adopted. To solve the problem of timeliness and optimization, each control variable is divided into 9 gradients, and there are 729 control schemes in the control sequence. Through contrast pursuit simulation experiments, it is verified that the maneuver decision method combining the dynamic relational weight algorithm and moving time strategy has a better accuracy and real-time performance. In the case of using prediction and not using prediction, the adaptive countermeasure simulation is carried out with the current more advanced Bayesian inference maneuvering decision-making scheme. The results show that the UCAV maneuvering decision-making ability combined with accurate prediction is better.

show abstract

Research on Air Confrontation Maneuver Decision-Making Method Based on Reinforcement Learning

Cited by 59 publications

References 11 publications

Application of Deep Reinforcement Learning in Maneuver Planning of Beyond-Visual-Range Air Combat

Application of Deep Reinforcement Learning in Maneuver Planning of Beyond-Visual-Range Air Combat

Autonomous Control of Unmanned Aerial Vehicles

Moving Time UCAV Maneuver Decision Based on the Dynamic Relational Weight Algorithm and Trajectory Prediction

Contact Info

Product

Resources

About