A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines

Waxenegger-Wilfing, Günther; Dresia, Kai; Deeken, Jan C.; Oschwald, Michael

doi:10.1109/taes.2021.3074134

Cited by 17 publications

(7 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hardware upgrades are currently being undertaken at the test facility that will allow the implementation of these methods into the engine control algorithm. 38…”

Section: Discussionmentioning

confidence: 99%

Forecasting thermoacoustic instabilities in liquid propellant rocket engines using multimodal Bayesian deep learning

Sengupta

Waxenegger-Wilfing

Martin

et al. 2022

International Journal of Spray and Combustion Dynamics

View full text Add to dashboard Cite

We present a method that combines multiple sensory modalities in a rocket thrust chamber to predict impending thermoacoustic instabilities with uncertainties. This is accomplished by training an autoregressive Bayesian neural network model that forecasts the future amplitude of the dynamic pressure time series, using multiple sensor measurements (injector pressure/ temperature measurements, static chamber pressure, high-frequency dynamic pressure measurements, high-frequency OH* chemiluminescence measurements) and future flow rate control signals as input. The method is validated using experimental data from a representative cryogenic research thrust chamber. The Bayesian nature of our algorithms allows us to work with a dataset whose size is restricted by the expense of each experimental run, without making overconfident extrapolations. We find that the networks are able to accurately forecast the evolution of the pressure amplitude and anticipate instability events on unseen experimental runs 500 milliseconds in advance. We compare the predictive accuracy of multiple models using different combinations of sensor inputs. We find that the high-frequency dynamic pressure signal is particularly informative. We also use the technique of integrated gradients to interpret the influence of different sensor inputs on the model prediction. The negative log-likelihood of data points in the test dataset indicates that prediction uncertainties are well-characterized by our model and simulating a sensor failure event results in a dramatic increase in the epistemic component of the uncertainty, as would be expected when a Bayesian method encounters unfamiliar, out-of-distribution inputs.

show abstract

“…Hardware upgrades are currently being undertaken at the test facility that will allow the implementation of these methods into the engine control algorithm. 38…”

Section: Discussionmentioning

confidence: 99%

Forecasting thermoacoustic instabilities in liquid propellant rocket engines using multimodal Bayesian deep learning

Sengupta

Waxenegger-Wilfing

Martin

et al. 2022

International Journal of Spray and Combustion Dynamics

View full text Add to dashboard Cite

show abstract

“…It demonstrated that the designed controller can reduce the overshoot of thrust, as well as the pressure and mixture ratio. In the work by Waxenegger-Wilfing et al, 23 a reinforcement learning (RL) approach for the optimal control of the start-up process of a liquid rocket engine was presented. The method can track different steady-state operating points.…”

Section: Introductionmentioning

confidence: 99%

Throttling simulation and control of liquid oxygen/liquid methane rocket engine pressurized by the electric pump

Hu,

Wu,

Liang

et al. 2024

Proceedings of the Institution of Mechanical Engineers, Part I:

View full text Add to dashboard Cite

This article presented simulation and control of a liquid oxygen/liquid methane variable thrust rocket engine pressurized by the electric pump in the throttling process. To get a proper injection pressure drop, the variation of the injection area was investigated and determined. In the control scheme, the combustion pressure and mixture ratio were the targets. To simplify the control scheme, the pressure and mixture ratio were correlated to the oxygen mass flow rate and methane mass flow rate, respectively. By controlling the propellant mass flow rate, the target engine thrust was reached. In addition, the influence of mass flow rate measurement points on the control effect was analyzed. The simulation results showed that the designed controller can reduce the overshoot of the critical engine parameters. Therefore, the maximum temperature of the thrust chamber wall after the regenerative cooling was decreased. Consequently, the thrust control errors were within 3% except for the points where the thrust changed suddenly, which can meet the mission requirements.

show abstract

“…Among machine learning approaches, reinforcement learning (RL) has demonstrated unprecedented capabilities in solving decision-making problems [2], which is key to intelligently behave in previously unexplored dynamic environments. Remarkable progress has been registered in developing RL algorithms for various robotic applications including, but not limited to, manipulation [3], navigation [4], [5], tracking [6], path planning [7], and control [8], [9], [10], [11].…”

Section: Introductionmentioning

confidence: 99%

Learning-Based Navigation and Collision Avoidance Through Reinforcement for UAVs

Azzam

Chehadeh

Hay

et al. 2024

IEEE Trans. Aerosp. Electron. Syst.

View full text Add to dashboard Cite

Reinforcement learning (RL) has been proven to enable the automation of tasks involving complex sequential decision-making. The simulation to reality (sim2real) gap, however, poses a major challenge in most engineering applications. In this work, we propose a learning approach combining RL based navigation and collision avoidance scheme with low-level advanced control to bridge the sim2real gap for unmanned aerial vehicle (UAV) applications. The proposed approach puts the RL agent at the top of the control hierarchy to focus on behavioral intelligence. We demonstrate the transferability of the RL policy trained in simulation to a real UAV without randomization of the system's dynamic parameters. The direct transfer is enabled by (1) the use of deep neural networks with the modified relay feedback test (DNN-MRFT) to identify the parameters of the UAV, and (2) formulating a reward function to penalize excessive actor actions. Particularly, the RL agent generates high-level velocity actions to achieve the sought task, while the low-level controller minimizes any unwanted disturbances and model discrepancies. The proposed approach has been tested and validated using computer simulations and real-world experiments. The real-world experimental results demonstrated the agent's capability to achieve the navigation task with a 90% success rate. The experimental results can be found in this video: https://youtu.be/I1BF4mhJLLs.

show abstract

A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines

Cited by 17 publications

References 32 publications

Forecasting thermoacoustic instabilities in liquid propellant rocket engines using multimodal Bayesian deep learning

Forecasting thermoacoustic instabilities in liquid propellant rocket engines using multimodal Bayesian deep learning

Throttling simulation and control of liquid oxygen/liquid methane rocket engine pressurized by the electric pump

Learning-Based Navigation and Collision Avoidance Through Reinforcement for UAVs

Contact Info

Product

Resources

About