Guidance law of interceptors against a high-speed maneuvering target based on deep Q-Network

Abstract

This paper proposes a novel guidance law for intercepting a high-speed maneuvering target based on deep reinforcement learning, which mainly includes the interceptor–target relative motion model and value function approximation model based on deep Q-Network (DQN) with prioritized experience replay. First, a method called prioritized experience replay is applied to extract more efficient samples and reduce the training time. Second, to cope with the discrete action space of DQN, a normal acceleration is introduced to the state space, and the normal acceleration rate is chosen as the action. Then, the continuous normal acceleration command is obtained using numerical integral method. Third, to make the line-of-sight (LOS) rate converge rapidly, the reward function whose absolute value tends to zero has been constructed. Finally, compared with proportional navigation guidance (PNG) and the Q-Learning-based guidance law (QLG), the simulation experiments are implemented to intercept high-speed maneuvering targets at different acceleration policies. Simulation results demonstrate that the proposed DQN-based guidance law (DQNG) can obtain continuous acceleration command, make the LOS rate converge to zero rapidly, and hit the maneuvering targets using only the LOS rate. It also confirms that DQNG can realize the parallel-like approach and improve the interception performance of the interceptor to high-speed maneuvering targets. The proposed DQNG also has the advantages of avoiding the complicated formula derivation of traditional guidance law and eliminates the acceleration buffeting.

Optimal Penetration Guidance Law for High-Speed Vehicles against an Interceptor with Modified Proportional Navigation Guidance

“…Near space high-speed vehicle technology is an important milestone in the history of modern weapons and equipment. As the commanding height of science and technology, it greatly enriches the content of attack-defense confrontation in near space [1][2][3][4]. Aiming at the optimal penetration guidance problem of high-speed vehicles against a modified proportional guidance interceptor, this paper obtains the analytical solution of an optimal penetration strategy by introducing Hamilton's principle [5][6][7].…”

A Policy-Reuse Algorithm Based on Destination Position Prediction for Aircraft Guidance Using Deep Reinforcement Learning

“…However, the orientation of the landing platform is not considered, and vertical velocity control is not included in the action set. A DRL-based guidance law is proposed in [15] to deal with maneuvering of high-speed targets. Based on the DQN algorithm, a relative-motion model is established and reward function is designed that can obtain continuous acceleration commands, make the LOS rate converge to zero rapidly, and hit the maneuvering target using only the LOS rate.…”