Flowsheet generation through hierarchical reinforcement learning and graph neural networks

Abstract

Process synthesis experiences a disruptive transformation accelerated by artificial intelligence. We propose a reinforcement learning algorithm for chemical process design based on a state-of-the-art actor-critic logic. Our proposed algorithm represents chemical processes as graphs and uses graph convolutional neural networks to learn from process graphs. In particular, the graph neural networks are implemented within the agent architecture to process the states and make decisions. We implement a hierarchical and hybrid decision-making process to generate flowsheets, where unit operations are placed iteratively as discrete decisions and corresponding design variables are selected as continuous decisions. We demonstrate the potential of our method to design economically viable flowsheets in an illustrative case study comprising equilibrium reactions, azeotropic separation, and recycles. The results show quick learning in discrete, continuous, and hybrid action spaces. The method is predestined to include large action-state spaces and an interface to process simulators in future research.

Toward automatic generation of control structures for process flow diagrams with large language models

“…In our previous work, we showed already that flowsheets and flow information can be represented as graphs 42 . In addition, we already leveraged graph neural networks to learn from these flowsheet graphs 42 . These technologies are also promising in the context of the prediction of decentralized control structures. Hybrid AI solutions: Chemical engineers have developed fundamental principles of modeling and control.…”

“…29,31 Process diagrams (e.g., PFDs or P&IDs) of chemical plants can be represented as directed graphs. 31,42 Unit operations and control units can be illustrated as nodes in the graph, while material streams and signals are directed edges connecting the nodes. Figure 2 shows an illustrative example process containing a reactor with level control and a recycle loop with flow control.…”

“…42 In addition, we already leveraged graph neural networks to learn from these flowsheet graphs. 42 These technologies are also promising in the context of the prediction of decentralized control structures.…”

Outlook: How I Learned to Love Machine Learning (A Personal Perspective on Machine Learning in Process Systems Engineering)

“…Specifically, PSE can play a key role in finding suitable data representations for molecules, chemical reactions, dynamical systems, flowsheets, and expert logic (and connections between them); such representations can then be fed to ML tools to conduct diverse tasks. For instance, recent work by the PSE community has explored data representations and ML models to predict molecular properties. ,, Recent work by the PSE community has also developed data representations of flowsheets as graphs and text-strings (analogous to SMILES strings) and has used these to train ML models that can automatically synthesize flowsheets. − ML tools such as physics-informed neural networks and physics-constrained neural networks also provide hybrid modeling capabilities that allow PSE researchers to fuse data-driven and physical models in new ways. − The PSE community has also developed new control, optimization, scheduling, and experimental design formulations that make use of ML techniques. ,− All this work is a clear example of how PSE leverages tools of ML to come up with innovative abstractions that facilitate discovery and decision-making. It is important to highlight that the PSE community was an early adopter of ML tools such as neural networks (going back to the 1980s and 1990s), but this early adoption was not as widespread.…”

“…The generation and standardization of data sets in different applications is essential; along these lines, I think that it is important to find unifying abstractions that can represent diverse systems under a common framework. For instance, there has been recent work in the PSE community that has proposed to represent molecules, flowsheets, and infrastructures as graphs. ,− As another example, there is recent work that aims to unify notions of closed-loop learning, system identification, and model predictive control …”