Reinforcement Learning for Robust Trajectory Design of Interplanetary Missions

Zavoli, Alessandro; Federici, Lorenzo

doi:10.2514/1.g005794

Cited by 69 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To fully evaluate the performance of the proposed scheduling networks, GA, DQN and another DRL algorithm that widely used in several literatures [34,35], i.e. Proximal Policy Optimization (PPO), are used to optimize the imaging sequence under same conditions in "Training".…”

Section: Comparison With Other Algorithmsmentioning

confidence: 99%

Autonomous imaging scheduling networks of small celestial bodies flyby based on deep reinforcement learning

Hu,

Wu,

Song

et al. 2024

Complex Intell. Syst.

View full text Add to dashboard Cite

During the flyby mission of small celestial bodies in deep space, it is hard for spacecraft to take photos at proper positions only rely on ground-based scheduling, due to the long communication delay and environment uncertainties. Aimed at imaging properly, an autonomous imaging policy generated by the scheduling networks that based on deep reinforcement learning is proposed in this paper. A novel reward function with relative distance variation in consideration is designed to guide the scheduling networks to obtain higher reward. A new part is introduced to the reward function to improve the performance of the networks. The robustness and adaptability of the proposed networks are verified in simulation with different imaging missions. Compared with the results of genetic algorithm (GA), Deep Q-network (DQN) and proximal policy optimization (PPO), the reward obtained by the trained scheduling networks is higher than DQN and PPO in most imaging missions and is equivalent to that of GA but, the decision time of the proposed networks after training is about six orders of magnitude less than that of GA, with less than 1e−4 s. The simulation and analysis results indicate that the proposed scheduling networks have great potential in further onboard application.

show abstract

Section: Comparison With Other Algorithmsmentioning

confidence: 99%

Autonomous imaging scheduling networks of small celestial bodies flyby based on deep reinforcement learning

Hu,

Wu,

Song

et al. 2024

Complex Intell. Syst.

View full text Add to dashboard Cite

show abstract

“…Federici et al 13 studied Deep Learning techniques for autonomous spacecraft guidance during proximity operations. Zavioli and Federici also studied RL for interplanetary trajectory design in 14 . More recently, an advanced machine learning framework called “meta-reinforcement learning” has been studied by Federici et al 15 , Scorsoglio et al 16 .…”

Section: Introductionmentioning

confidence: 99%

Stochastic learning and extremal-field map based autonomous guidance of low-thrust spacecraft

Singh

Junkins

2022

Sci Rep

View full text Add to dashboard Cite

A supervised stochastic learning method called the Gaussian Process Regression (GPR) is used to design an autonomous guidance law for low-thrust spacecraft. The problems considered are both of the time- and fuel-optimal regimes and a methodology based on “perturbed back-propagation” approach is presented to generate optimal control along neighboring optimal trajectories which form the extremal bundle constituting the training data-set. The use of this methodology coupled with a GPR approximation of the spacecraft control via prediction of the costate n-tuple or the primer vector respectively for time- and fuel-optimal trajectories at discrete time-steps is demonstrated to be effective in designing an autonomous guidance law using the open-loop bundle of trajectories to-go. The methodology is applied to the Earth-3671 Dionysus time-optimal interplanetary transfer of a low-thrust spacecraft with off-nominal thruster performance and the resulting guidance law is evaluated under different design parameters using case-studies. The results highlight the utility and applicability of the proposed framework with scope for further improvements.

show abstract

“…Also, the use of machine learning techniques is being studied for spacecraft guidance and control [21] under uncertainty, e.g. using neural networks [22,23] or reinforcement learning [24,25].…”

mentioning

confidence: 99%

Robust Space Trajectory Design Using Belief Optimal Control

Greco

Campagnola

Vasile

2022

Journal of Guidance, Control, and Dynamics

View full text Add to dashboard Cite

This paper presents a novel approach to the robust solution of optimal impulsive control problems under aleatory and epistemic uncertainty. The novel approach uses belief Markov decision processes to reformulate the control problem in terms of uncertainty distributions, called beliefs, rather than the realisations of the system states. This formulation leads to the definition of a Belief Optimal Control problem where the cost function and constraints are functions of the uncertainty distributions. The control formulation encompasses orbit determination arcs as well. The belief optimisation is solved with a shooting-like transcription and a nonlinear programming solver to optimise the resulting discretised problem. Both aleatory and epistemic uncertainties are propagated with a non-intrusive polynomial expansion to capture the nonlinearities of the dynamics. Finally, this new approach is applied to the robust optimisation of a flyby trajectory of the Europa Clipper mission in a scenario characterised by knowledge, execution and observation uncertainty.

show abstract

Reinforcement Learning for Robust Trajectory Design of Interplanetary Missions

Cited by 69 publications

References 32 publications

Autonomous imaging scheduling networks of small celestial bodies flyby based on deep reinforcement learning

Autonomous imaging scheduling networks of small celestial bodies flyby based on deep reinforcement learning

Stochastic learning and extremal-field map based autonomous guidance of low-thrust spacecraft

Robust Space Trajectory Design Using Belief Optimal Control

Contact Info

Product

Resources

About