Searching for optimal process routes: A reinforcement learning approach

Khan, Ahmad; Lapkin, Alexei

doi:10.1016/j.compchemeng.2020.107027

Cited by 30 publications

(16 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…26 In order to improve the efficiency of route design the machine learning method of reinforcement learning has been proposed for the application in synthesis planning. 18,[26][27][28] Reinforcement learning (RL) mimics how an intelligent 'decision maker' takes multi-step actions within a specific problem environment to maximise/minimise the cumulative rewards/penalties of the actions. 29 In synthetic planning, the selection of each reaction step within a path is a decision making step.…”

Section: Introductionmentioning

confidence: 99%

Hybridizing organic chemistry and synthetic biology reaction networks for optimizing synthesis routes

Zhang

Lapkin

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Computer assisted synthesis planning (CASP) accelerates the development of organic synthesis routes of pharmaceuticals and industrial chemicals. CASP tools are generally developed on the rules or data of synthetic chemistry which include some enzymatic reactions. However, synthetic biology offers a new degree of freedom through the potential to engineer new synthetic steps. In this work, we present a method to hybridise conventional organic synthesis and synthetic biology to guide synthesis planning. A section of organic reactions from Reaxys® database was combined with metabolic reactions from KEGG database as reactions pools. The combined database was used to assemble synthetic pathways from multiple building blocks to a target molecule. The routes assembly was performed using reinforcement learning, which was adapted to learn the values of molecular structures in synthesis planning, and to develop a policy model to suggest near-optimal multi-step synthesis route choices from the available reactions pool. To quantify the added value of synthetic biology in the hybrid routes, three policy model ‘decision makers’ were developed from the organic, biological and hybrid reactions pools respectively. The near-optimal synthetic routes planned from the three reactions pools were evaluated and compared to discuss the benefits of the hybrid synthetic chemistry plus synthetic biology decision space in reaction routes optimisation.

show abstract

Section: Introductionmentioning

confidence: 99%

Hybridizing organic chemistry and synthetic biology reaction networks for optimizing synthesis routes

Zhang

Lapkin

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, important first steps have been made towards using RL to synthesize novel process flowsheets [34][35][36][37][38][39] . Midgley 34 introduced the "Distillation Gym", an environment in which distillation trains for non-azeotropic mixtures are generated by a soft-actor-critic RL agent and simulated in the opensource process simulator COCO.…”

Section: Introductionmentioning

confidence: 99%

Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks

Stops¹,

Roel²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Process synthesis experiences a disruptive transformation accelerated by digitization and artificial intelligence. We propose a reinforcement learning algorithm for chemical process design based on a state-of-the-art actor-critic logic. Our proposed algorithm represents chemical processes as graphs and uses graph convolutional neural networks to learn from process graphs. In particular, the graph neural networks are implemented within the agent architecture to process the states and make decisions. Moreover, we implement a hierarchical and hybrid decision-making process to generate flowsheets, where unit operations are placed iteratively as discrete decisions and corresponding design variables are selected as continuous decisions. We demonstrate the potential of our method to design economically viable flowsheets in an illustrative case study comprising equilibrium reactions, azeotropic separation, and recycles. The results show quick learning in discrete, continuous, and hybrid action spaces. Due to the flexible architecture of the proposed reinforcement learning agent, the method is predestined to include large action-state spaces and an interface to process simulators in future research.

show abstract

“…Among rare exceptions are Zhou et al [25], who employed RL to set experimental conditions for the optimization of chemical reactions. Khan and Lapkin [26] used a RL approach to identify promising processing routes in hydrogen production.…”

Section: Introductionmentioning

confidence: 99%

Automated synthesis of steady-state continuous processes using reinforcement learning

Göttl

Grimm

Burger

2021

Front. Chem. Sci. Eng.

View full text Add to dashboard Cite

Automated flowsheet synthesis is an important field in computer-aided process engineering. The present work demonstrates how reinforcement learning can be used for automated flowsheet synthesis without any heuristics or prior knowledge of conceptual design. The environment consists of a steady-state flowsheet simulator that contains all physical knowledge. An agent is trained to take discrete actions and sequentially build up flowsheets that solve a given process problem. A novel method named SynGameZero is developed to ensure good exploration schemes in the complex problem. Therein, flowsheet synthesis is modelled as a game of two competing players. The agent plays this game against itself during training and consists of an artificial neural network and a tree search for forward planning. The method is applied successfully to a reaction-distillation process in a quaternary system.

show abstract

Searching for optimal process routes: A reinforcement learning approach

Cited by 30 publications

References 14 publications

Hybridizing organic chemistry and synthetic biology reaction networks for optimizing synthesis routes

Hybridizing organic chemistry and synthetic biology reaction networks for optimizing synthesis routes

Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks

Automated synthesis of steady-state continuous processes using reinforcement learning

Contact Info

Product

Resources

About