Missile guidance with assisted deep reinforcement learning for head-on interception of maneuvering target

Abstract

In missile guidance, pursuit performance is seriously degraded due to the uncertainty and randomness in target maneuverability, detection delay, and environmental noise. In many methods, accurately estimating the acceleration of the target or the time-to-go is needed to intercept the maneuvering target, which is hard in an environment with uncertainty. In this paper, we propose an assisted deep reinforcement learning (ARL) algorithm to optimize the neural network-based missile guidance controller for head-on interception. Based on the relative velocity, distance, and angle, ARL can control the missile to intercept the maneuvering target and achieve large terminal intercept angle. To reduce the influence of environmental uncertainty, ARL predicts the target’s acceleration as an auxiliary supervised task. The supervised learning task improves the ability of the agent to extract information from observations. To exploit the agent’s good trajectories, ARL presents the Gaussian self-imitation learning to make the mean of action distribution approach the agent’s good actions. Compared with vanilla self-imitation learning, Gaussian self-imitation learning improves the exploration in continuous control. Simulation results validate that ARL outperforms traditional methods and proximal policy optimization algorithm with higher hit rate and larger terminal intercept angle in the simulation environment with noise, delay, and maneuverable target.

Intelligent maneuver strategy for hypersonic vehicles in three-player pursuit-evasion games via deep reinforcement learning

“…In a different study ( Jiang et al, 2022 ), the problem was reformulated as a Markov decision process (MDP), and an Actor-Critic (AC) framework-based DRL algorithm was used to solve it to suggest the anti-interception guiding law. To intercept the moving target, Li et al (2022) somewhat enhanced the reinforcement learning algorithm. The ideal attitude-tracking problem for HVs during the reentry phase ( Zhao et al, 2022 ) was solved using the RL algorithm.…”

Intelligent Maneuver Strategy for a Hypersonic Pursuit-Evasion Game Based on Deep Reinforcement Learning

“…The intelligent game maneuver adopts a closed-loop maneuver scheme of "interceptor movement-situational awareness-maneuver strategy generation-maneuver control implementation" that realizes timely maneuvering to increase miss distance and increase evasion probability. The key to intelligent game maneuver lies in the selection of intelligent algorithms Among the intelligent algorithms associated with hypersonic aircraft, deep learning (DL)and reinforcement learning (RL) are the first to bear the brunt [21][22][23][24][25][26][27][28][29][30][31][32]. Due to its strong nonlinear fitting ability, the deep neural network (DNN) in DL has been widely used in the PE problems of hypersonic aircraft [21][22][23].…”

“…Among these, the most prevalent study [21] resolves the tension between the accuracy and speed of the IPP by building an IPP neural network model after using the ballistic model to create training data. And the algorithms of reinforcement learning, especially deep reinforcement learning (DRL), provide a new approach to the design of HVs' evasion strategies [24][25][26][27][28][29][30][31][32]. As an unsupervised heuristic algorithm without an accurate model, RL and DRL can generate actions based on the interaction with the environment, that is, conduct intelligent maneuvering games based on both attack and defense sides.…”

“…Another study [27] transformed the problem into a Markov decision process (MDP) and proposed the anti-interception guidance law utilizing a DRL algorithm consisting of an actor-critic framework to solve it. The research [28] improved the reinforcement learning algorithm to a certain extent to achieve the interception of the maneuvering target. In the study [29], the RL was used to solve the optimal attitude-tracking problem for hypersonic vehicles in the reentry phase.…”

Online Trajectory Replan for Gliding Vehicle Considering Terminal Velocity Constraint

“…By converting the guidance problem into a reinforcement learning problem, neural networks were trained within the available energy consumption to generate optimal trajectories and guidance command. Li et al [29] proposed a deep reinforcement learning-based guidance law for trajectory generation considering uncertain environments. Another approach is to utilize predictive methods.…”