Two-Sided, Genetics-Based Learning to Discover Novel Fighter Combat Maneuvers

Smith, Robert E.; Dike, B. A.; Ravichandran, B.; El-Fallah, A.; Mehra, Raman K.

doi:10.1007/3-540-45365-2_24

Cited by 3 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to the core methods, it can be roughly divided into two categories. One method is based on traditional mathematical solutions, such as the influence diagram method [5], genetic optimization algorithm [6], game theory [7,8], and so on. This type of method has a clear mathematical expression, but it is difficult to solve and is effective only for simple air combat environments.…”

Section: Introductionmentioning

confidence: 99%

Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

Ruan,

Qin,

Wang

et al. 2024

Mathematics

View full text Add to dashboard Cite

To adapt to the development trend of intelligent air combat, it is necessary to research the autonomous generation of maneuvering decisions for unmanned combat aerial vehicles (UCAV). This paper presents a maneuver decision-making method for UCAV based on a hybridization of deep Q-network (DQN) and extended Kalman filtering (EKF). Firstly, a three-dimensional air combat simulation environment is constructed, and a flight motion model of UCAV is designed to meet the requirements of the simulation environment. Secondly, we evaluate the current situation of UCAV based on their state variables in air combat, for further network learning and training to obtain the optimal maneuver strategy. Finally, based on the DQN, the system state equation is constructed using the uncertain parameter values of the current network, and the observation equation of the system is constructed using the parameters of the target network. The optimal parameter estimation value of the DQN is obtained by iteratively updating the solution through EKF. Simulation experiments have shown that this autonomous maneuver decision-making method hybridizing DQN with EKF is effective and reliable, as it can eliminate the opponent and preserve its side.

show abstract

Section: Introductionmentioning

confidence: 99%

Research on Deep Q-Network Hybridization with Extended Kalman Filter in Maneuvering Decision of Unmanned Combat Aerial Vehicles

Ruan,

Qin,

Wang

et al. 2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…(iii) An alternate freeze game framework is proposed to deal with nonstationarity, which can be used to solve the problem of variable opponent strategies in RL. e league system is adopted to select an agent with the highest performance to avoid the red queen effect [45].…”

Section: Introductionmentioning

confidence: 99%

Improving Maneuver Strategy in Air Combat by Alternate Freeze Games with a Deep Reinforcement Learning Algorithm

Wang

et al. 2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In a one-on-one air combat game, the opponent’s maneuver strategy is usually not deterministic, which leads us to consider a variety of opponent’s strategies when designing our maneuver strategy. In this paper, an alternate freeze game framework based on deep reinforcement learning is proposed to generate the maneuver strategy in an air combat pursuit. The maneuver strategy agents for aircraft guidance of both sides are designed in a flight level with fixed velocity and the one-on-one air combat scenario. Middleware which connects the agents and air combat simulation software is developed to provide a reinforcement learning environment for agent training. A reward shaping approach is used, by which the training speed is increased, and the performance of the generated trajectory is improved. Agents are trained by alternate freeze games with a deep reinforcement algorithm to deal with nonstationarity. A league system is adopted to avoid the red queen effect in the game where both sides implement adaptive strategies. Simulation results show that the proposed approach can be applied to maneuver guidance in air combat, and typical angle fight tactics can be learnt by the deep reinforcement learning agents. For the training of an opponent with the adaptive strategy, the winning rate can reach more than 50%, and the losing rate can be reduced to less than 15%. In a competition with all opponents, the winning rate of the strategic agent selected by the league system is more than 44%, and the probability of not losing is about 75%.

show abstract