Autonomous closed-loop guidance using reinforcement learning in a low-thrust, multi-body dynamical environment

LaFarge, Nicholas B.; Miller, Daniel; Howell, Kathleen C.; Linares, Richard

doi:10.1016/j.actaastro.2021.05.014

Cited by 37 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, neural network controller trained by RL augments the robustness of existing control methods, so as to deal with complex constraints and environmental uncertainties in spacecraft rendezvous guidance, 21 proximity operations, 22 and onboard applications for low-thrust spacecraft. 23 These studies have shown that RL can improve the performance of conventional methods in controlling powered vehicles. However, the unique characteristics of unpowered HGVs make it more challenging to control them using RL.…”

Section: Introductionmentioning

confidence: 99%

Integrated entry guidance with no-fly zone constraint using reinforcement learning and predictor-corrector technique

Gao,

Zhou,

Chen

2024

Proceedings of the Institution of Mechanical Engineers, Part G:

View full text Add to dashboard Cite

This paper presents an integrated entry guidance law for hypersonic glide vehicles with no-fly zone constraint. Existing methods that employ predictor-corrector technique and lateral guidance logic for both guidance and avoidance, may have limitations in response time and maneuverability when facing sudden threats, because the guidance cycle is limited by computational efficiency and the bank angle magnitude cannot be adjusted according to the urgency of the avoidance. To overcome these challenges, the proposed method divides the entry process into safe flight stages and no-fly zone avoidance stages, and introduces reinforcement learning to develop an intelligent avoidance strategy for the latter. This division reduces the complexity of the learning problem by restricting the state space and increases the applicability in the presence of multiple no-fly zones. The trained avoidance strategy can directly output continuous bank angle command through a single forward calculation, considering both guidance and avoidance requirements. This enables the full utilization of the vehicle’s maneuverability and supports a high command update frequency to effectively handle threats. Additionally, a network trained via supervised learning is employed to generate reference commands, accelerating the training convergence of reinforcement learning. Simulation results demonstrate the effectiveness of the proposed guidance law, highlighting its high computational efficiency, command stability, and robustness. Importantly, the approach offers convenience in extending to multiple no-fly zones and accommodating vast initial state spaces.

show abstract

Section: Introductionmentioning

confidence: 99%

Integrated entry guidance with no-fly zone constraint using reinforcement learning and predictor-corrector technique

Gao,

Zhou,

Chen

2024

Proceedings of the Institution of Mechanical Engineers, Part G:

View full text Add to dashboard Cite

show abstract

“…Over the past decades, periodic orbits derived from the restricted three-body problem (RTBP) have been extensively employed as nominal trajectories in different deep space missions. To counteract the adverse effects of diverse perturbations in real mission environments, various station-keeping strategies have been designed specifically for periodic orbits to achieve long-term orbital maintenance [1][2][3][4]. Regarding the geometric symmetry of a periodic orbit, the x-axis crossing control strategy was developed to target certain conditions when intersecting the orbit's symmetry/near-symmetry plane [5].…”

Section: Introductionmentioning

confidence: 99%

A high-order target phase approach for the station-keeping of periodic orbits

Fu,

Baresi,

Armellin

2024

Astrodyn

View full text Add to dashboard Cite

A novel high-order target phase approach (TPhA) for the station-keeping of periodic orbits is proposed in this work. The key elements of the TPhA method, the phase-angle Poincare map and high-order maneuver map, are constructed using differential algebra (DA) techniques to determine station-keeping epochs and calculate correction maneuvers. A stochastic optimization framework tailored for the TPhA-based station-keeping process is leveraged to search for fuel-optimal and error-robust TPhA parameters. Quasi-satellite orbits (QSOs) around Phobos are investigated to demonstrate the efficacy of TPhA in mutli-fidelity dynamical models. Monte Carlo simulations demonstrated that the baseline QSO of JAXA’s Martian Moons eXploration (MMX) mission could be maintained with a monthly maneuver budget of approximately 1 m/s.

show abstract

“…For example, the exploration vehicle Orion is equipped with a far larger autonomy and automation with respect to the Space Shuttle or the International Space Station, as it is required to autonomously perform rendezvous and docking operations or the deorbit burn [8]. Therefore, the design of future space vehicles must guarantee the autonomous execution of all GNC functions [9]. This involves the necessity to address two compelling requirements: first, develop a guidance algorithm able to manage a complex dynamical environment, which is nonlinear and highly perturbed, resulting in the inadequacy of Keplerian-based algorithms [7]; second, determine suitable techniques that can be implemented on an onboard software with limited computational resources.…”

Section: Introductionmentioning

confidence: 99%

Low-Thrust Nonlinear Orbit Control for Very Low Lunar Orbits

Leonardi,

Pontani,

Carletta

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

In the next decades, both space agencies and private competitors are targeting the lunar environment as a scientific and technological resource for future space missions. In particular, the confirmed existence of water-ice deposits in the vicinity of the poles (predominantly the south pole) makes polar or near-polar low lunar orbits attractive for the purpose of designing space missions that could search for suitable Lunar base sites. However, traveling very-low-altitude orbits is very challenging, as they are strongly perturbed by the Moon’s gravity field as well as third- and fourth-body effects due to the Earth and the Sun. Several studies demonstrate that these orbits are expected to impact the lunar surface in a few months. Therefore, the definition and implementation of an effective station-keeping strategy represents a crucial issue in order to extend satellites’ lifetime. In this paper, a feedback nonlinear control law is employed in order to perform corrective maneuvers aimed at keeping the state of the satellite within acceptable margins. The satellite is assumed to be equipped with a steerable and throttleable low-thrust propulsion system. The control law is based on the Lyapunov stability theory and does not require any reference path to track, with a considerable decrease in the computational cost. The proposed real-time control law includes control saturation, related to the maximum available thrust magnitude, and is developed employing modified equinoctial elements, in order to avoid singularities and extend its range of application. Finally, the strategy at hand is tested in the presence of all the relevant perturbations (i.e., harmonics of the selenopotential, third- and fourth-body effects) in order to show its effectiveness and efficiency.

show abstract

Autonomous closed-loop guidance using reinforcement learning in a low-thrust, multi-body dynamical environment

Cited by 37 publications

References 23 publications

Integrated entry guidance with no-fly zone constraint using reinforcement learning and predictor-corrector technique

Integrated entry guidance with no-fly zone constraint using reinforcement learning and predictor-corrector technique

A high-order target phase approach for the station-keeping of periodic orbits

Low-Thrust Nonlinear Orbit Control for Very Low Lunar Orbits

Contact Info

Product

Resources

About