Reinforcement Learning – Overview of recent progress and implications for process control

Shin, Jeong‐Hun; Badgwell, Thomas A.; Liu, Kuang-Hung; Lee, Jay

doi:10.1016/j.compchemeng.2019.05.029

Cited by 204 publications

(80 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A current focus in RL research is centered around creating systems that learn more efficiently with remarkable recent advances in causal discovery (Zhu et al, 2019) and meta-learning (Co-Reyes et al, 2021). Looking ahead, the RL framework holds the potential to improve solutions across many complex chemical engineering problems, such as scheduling (Hubbs et al, 2020), control (Shin et al, 2019), and process optimization (Petsagkourakis et al, 2020). The framework is also a viable alternative to exact optimization-based approaches for large design spaces.…”

Section: Optimization Platforms For Integrated Multiscale Designmentioning

confidence: 99%

Paradigm Shift: The Promise of Deep Learning in Molecular Systems Engineering and Design

Alshehri

You

2021

Front. Chem. Eng.

View full text Add to dashboard Cite

The application of deep learning to a diverse array of research problems has accelerated progress across many fields, bringing conventional paradigms to a new intelligent era. Just as the roles of instrumentation in the old chemical revolutions, we reinforce the necessity for integrating deep learning in molecular systems engineering and design as a transformative catalyst towards the next chemical revolution. To meet such research needs, we summarize advances and progress across several key elements of molecular systems: molecular representation, property estimation, representation learning, and synthesis planning. We further spotlight recent advances and promising directions for several deep learning architectures, methods, and optimization platforms. Our perspective is of interest to both computational and experimental researchers as it aims to chart a path forward for cross-disciplinary collaborations on synthesizing knowledge from available chemical data and guiding experimental efforts.

show abstract

Section: Optimization Platforms For Integrated Multiscale Designmentioning

confidence: 99%

Paradigm Shift: The Promise of Deep Learning in Molecular Systems Engineering and Design

Alshehri

You

2021

Front. Chem. Eng.

View full text Add to dashboard Cite

show abstract

“…The objective of RL is to teach an agent, which could for example consist of an artificial neural network (ANN), to master a given task through repeated interactions with its environment [22,23]. RL is already applied in process engineering, however almost exclusively in process control [24]. Among rare exceptions are Zhou et al [25], who employed RL to set experimental conditions for the optimization of chemical reactions.…”

Section: Introductionmentioning

confidence: 99%

Automated synthesis of steady-state continuous processes using reinforcement learning

Göttl

Grimm

Burger

2021

Front. Chem. Sci. Eng.

View full text Add to dashboard Cite

Automated flowsheet synthesis is an important field in computer-aided process engineering. The present work demonstrates how reinforcement learning can be used for automated flowsheet synthesis without any heuristics or prior knowledge of conceptual design. The environment consists of a steady-state flowsheet simulator that contains all physical knowledge. An agent is trained to take discrete actions and sequentially build up flowsheets that solve a given process problem. A novel method named SynGameZero is developed to ensure good exploration schemes in the complex problem. Therein, flowsheet synthesis is modelled as a game of two competing players. The agent plays this game against itself during training and consists of an artificial neural network and a tree search for forward planning. The method is applied successfully to a reaction-distillation process in a quaternary system.

show abstract

“…The proposed RL approach in this article is a model‐based algorithm, because model‐free RL approaches require an excessive amount of data because complex nonlinear system dynamics and structural characteristics should be learned only from the data 58 . As discussed in References 5, 59, the model‐free RL has an exponential lower bound on the sample complexity while the model‐based RL has a polynomial sample complexity.…”

Section: Introductionmentioning

confidence: 99%

Model‐based reinforcement learning for nonlinear optimal control with practical asymptotic stability guarantees

Kim

Lee

2020

AIChE Journal

View full text Add to dashboard Cite

We propose a new reinforcement learning approach for nonlinear optimal control where the value function is updated as restricted to control Lyapunov function (CLF) and the policy is improved using a variation of Sontag's formula. The practical asymptotic stability of the closed‐loop system is guaranteed during the training and at the end of training without requiring an additional actor network and its update rule. For a single‐layer neural network (NN) with exact basis functions, the approximate function converges to the optimal value function, resulting in the optimal controller. When a deep NN is used, the level set shapes of the trained NN become similar to those of the optimal value function. Because Sontag's formula with CLF is equivalent to the optimal controller when the given CLF has the same level set shapes as the optimal value function, Sontag's formula with the trained NN provides a nearly optimal controller.

show abstract

Reinforcement Learning – Overview of recent progress and implications for process control

Cited by 204 publications

References 31 publications

Paradigm Shift: The Promise of Deep Learning in Molecular Systems Engineering and Design

Paradigm Shift: The Promise of Deep Learning in Molecular Systems Engineering and Design

Automated synthesis of steady-state continuous processes using reinforcement learning

Model‐based reinforcement learning for nonlinear optimal control with practical asymptotic stability guarantees

Contact Info

Product

Resources

About