Intelligent Online Multiconstrained Reentry Guidance Based on Hindsight Experience Replay

Abstract

Traditional guidance algorithms for hypersonic glide vehicles face the challenge of real-time requirements and robustness to multiple deviations or tasks. In this paper, an intelligent online multiconstrained reentry guidance is proposed to strikingly reduce computational burden and enhance the effectiveness with multiple constraints. First, the simulation environment of reentry including dynamics, multiconstraints, and control variables is built. Different from traditional decoupling methods, the bank angle command including its magnitude and sign is designed as the sole guidance variable. Secondly, a policy neural network is designed to output end-to-end guidance commands. By transforming the reentry process into a Markov Decision Process (MDP), the policy network can be trained by deep reinforcement learning (DRL). To address the sparse reward issue caused by multiconstraints, the improved Hindsight Experience Replay (HER) method is adaptively combined with Deep Deterministic Policy Gradient (DDPG) algorithm by transforming multiconstraints into multigoals. As a result, the novel training algorithm can realize higher utilization of failed data and improve the rate of convergence. Finally, simulations for typical scenes show that the policy network in the proposed guidance can output effective commands in much less time than the traditional method. The guidance is robust to initial bias, different targets, and online aerodynamic deviation.

Trajectory optimization for the diving segment of air-breathing supersonic vehicle with multi-objective

“…Jiang et al used deep reinforcement learning to optimize the trajectory of the dive section of the reentry glide vehicle, proving the effectiveness of neural networks in this aspect [4]. Liang et al proposed a two-stage trajectory planning method combining a parametric control profile and biased proportional guidance for trajectory optimization in the diving segment of the supersonic vehicle with landing velocity and Angle constraints [5].…”