Beyond-visual-range (BVR) engagement becomes more and more popular in the modern air battlefield. The key and difficulty for pilots in the fight is maneuver planning, which reflects the tactical decision-making capacity of the both sides and determinates success or failure. In this paper, we propose an intelligent maneuver planning method for BVR combat with using an improved deep Q network (DQN). First, a basic combat environment builds, which mainly includes flight motion model, relative motion model and missile attack model. Then, we create a maneuver decision framework for agent interaction with the environment. Basic perceptive variables are constructed for agents to form continuous state space. Also, considering the threat of each side missile and the constraint of airfield, the reward function is designed for agents to training. Later, we introduce a training algorithm and propose perceptional situation layers and value fitting layers to replace policy network in DQN. Based on long short-term memory (LSTM) cell, the perceptional situation layer can convert basic state to high-dimensional perception situation. The fitting layer does well in mapping action. Finally, three combat scenarios are designed for agent training and testing. Simulation result shows the agent can avoid the threat of enemy and gather own advantages to threat the target. It also proves the models and methods of agents are valid and intelligent air combat can be realized.

Trajectory prediction plays an important role in modern air combat. Aiming at the large degree of modern simplification, low prediction accuracy, poor authenticity and reliability of data sample in traditional methods, a trajectory prediction method based on HPSO-TPFENN neural network is established by combining with the characteristics of trajectory with time continuity. The time profit factor was introduced into the target function of Elman neural network, and the parameters of improved Elman neural network are optimized by using the hybrid particle swarm optimization algorithm (HPSO), and the HPSO-TPFENN neural network was constructed. An independent prediction method of three-dimensional coordinates is firstly proposed, and the trajectory prediction data sample including course angle and pitch angle is constructed by using true combat data selected in the air combat maneuvering instrument (ACMI), and the trajectory prediction model based on HPSO-TPFENN neural network is established. The precision and real-time performance of trajectory prediction model are analyzed through the simulation experiment, and the results show that the relative error in different direction is below 1%, and it takes about 42ms approximately to complete 595 consecutive prediction, indicating that the present model can accurately and quickly predict the trajectory of the target aircraft.

No abstract

Air target threat assessment is a key issue in air defense operations. Aiming at the shortcomings of traditional threat assessment methods, such as one-sided, subjective, and low-accuracy, a new method of air target threat assessment based on gray neural network model (GNNM) optimized by improved moth flame optimization (IMFO) algorithm is proposed. The model fully combines with excellent optimization performance of IMFO with powerful learning performance of GNNM. Finally, the model is trained and evaluated using the target threat database data. The simulation results show that compared with the GNNM model and the MFO-GNNM model, the proposed model has a mean square error of only 0.0012 when conducting threat assessment, which has higher accuracy and evaluates 25 groups of targets in 10 milliseconds, which meets real-time requirements. Therefore, the model can be effectively used for air target threat assessment.

Unmanned aerial vehicle (UAV) swarm cooperative decision-making has attracted increasing attentions because of its low-cost, reusable, and distributed characteristics. However, existing non-learningbased methods rely on small-scale, known scenarios, and cannot solve complex multi-agent cooperation problem in large-scale, uncertain scenarios. This paper proposes a hierarchical multi-agent reinforcement learning (HMARL) method to solve the heterogeneous UAV swarm cooperative decision-making problem for the typical suppression of enemy air defense (SEAD) mission, which is decoupled into two subproblems, i.e., the higher-level target allocation (TA) sub-problem and the lower-level cooperative attacking (CA) sub-problem. We establish a HMARL agent model, consisting of a multi-agent deep Q network (MADQN) based TA agent and multiple independent asynchronous proximal policy optimization (IAPPO) based CA agents. MADQN-TA agent can dynamically adjust the TA schemes according to the relative position. To encourage exploration and promote learning efficiency, the Metropolis criterion and inter-agent information exchange techniques are introduced. IAPPO-CA agent adopts independent learning paradigm, which can easily scale with the number of agents. Comparative simulation results validate the effectiveness, robustness, and scalability of the proposed method.INDEX TERMS UAV swarm; suppression of enemy air defense; deep reinforcement learning; multi-agent; hierarchical reinforcement learning