Low-Thrust Trajectory Design Using Closed-Loop Feedback-Driven Control Laws and State-Dependent Parameters

Abstract

Low-thrust many-revolution trajectory design and orbit transfers are becoming increasingly important with the development of high specific impulse, low-thrust engines. Closed-loop feedback-driven (CLFD) control laws can be used to solve these trajectory design problems with minimal computational cost and offer potential for autonomous guidance. However, they have user-defined parameters which limit their optimality. In this work, an actor-critic reinforcement learning framework is proposed to make the parameters of the Lyapunov-based Q-law state-dependent, ensuring the controller can adapt as the dynamics evolve during a transfer. The proposed framework should be independent of the particular CLFD control law and provides improved solutions for mission analysis. There is also potential for future on-board autonomous use, as trajectories are closed-form and can be generated without an initial guess. The current results focus on GTO-GEO transfers in Keplerian dynamics and later with eclipse and J 2 effects. Both time-optimal and mass-optimal transfers are presented, and the stability to uncertainties in orbit determination are discussed. The task of handling orbit perturbations is left to future work.

Near-Optimal Feedback Guidance for Low-Thrust Earth Orbit Transfers

“…The feedback laws (7) and Eq. ( 9) can be combined, to yield This final form of the feedback law ensures asymptotical stability and accounts for the limitations of the propulsion system.…”

Optimal Q-laws via reinforcement learning with guaranteed stability

“…Using a constant value works very well and has achieved satisfactory results in previous work. 38,39 However, improved results can be obtained when using an adaptive approach. Hence, starting with σ = 0.1, we explore until 50 consecutive iterations occur without improvement, then σ is reduced to 0.03.…”

“…The advantage of combining these two approaches allows one to eliminate the drawbacks from each approach: namely the sub-optimality of Lyapunov controllers and the unknown stability of RL methods. The authors demonstrated a prototype RL architecture and control approach in conference proceedings, 38,39 showing potential for space-borne applications. Here we present a state-dependent controller which enforces stability without compromising optimality through the Jacobian of the state-dependent parameters.…”

Reference governor for constrained spacecraft orbital transfers

“…The feedback control for orbital transfers has also been investigated, 2,[4][5][6] in particular, for missions that exploit the low thrust propulsion. For instance, the Q-law 4,7 has been proposed and extensively studied.…”