A guidance method for coplanar orbital interception based on reinforcement learning

Abstract

This paper investigates the guidance method based on reinforcement learning (RL) for the coplanar orbital interception in a continuous low-thrust scenario. The problem is formulated into a Markov decision process (MDP) model, then a welldesigned RL algorithm, experience based deep deterministic policy gradient (EBDDPG), is proposed to solve it. By taking the advantage of prior information generated through the optimal control model, the proposed algorithm not only resolves the convergence problem of the common RL algorithm, but also successfully trains an efficient deep neural network (DNN) controller for the chaser spacecraft to generate the control sequence. Numerical simulation results show that the proposed algorithm is feasible and the trained DNN controller significantly improves the efficiency over traditional optimization methods by roughly two orders of magnitude.

An AutoML based trajectory optimization method for long-distance spacecraft pursuit-evasion game

“…, and , the final state and costate can be obtained through integrating (1) and (8). The final position vectors should satisfy (3), final position costate should satisfy (12), final velocity costate should satisfy (10), and final Hamiltonian should satisfy (11).…”

“…Especially, Wu et al [7] proposed a method directly using DNNs to fast derive the optimal interception trajectories, learning optimal control law from datasets generated by traditional methods. George [8] presented a method in reinforcement learning manner, combining networks and evolutionary algorithm to solve optimal trajectories for interception and rendezvous scenarios with three different thrust configurations. Zeng et al [9] proposed a reinforcement learning method for coplanar orbital interception in long-distance scenarios, where an efficient DNN controller was trained to generate the optimal control sequence for spacecraft.…”

“…Thus, the costate normalization is employed to restrict on a unit 12-dimensional hypersphere. After defining new Lagrange multiplier , it is determined that the differential game, which is expressed by the performance index (2), the dynamics (1), the Euler-Lagrange equations (8), the optimal conditions (5) and ( 6), the Hamiltonian (4), and the shooting function (13), is homogeneous to the Lagrange multiplier. Therefore, multiplying the multiplier by a positive factor does not change the nature of this problem.…”

Two-Loop Acceleration Autopilot Design and Analysis Based on TD3 Strategy

“…In recent years, deep learning (DL) and reinforcement learning (RL) have become hot topics in artificial intelligence technology, providing new design options for aircraft guidance and control systems [13][14][15]. The principle of RL originates from the process of intelligent species learning new things.…”