Integration of machine learning and first principles models

Abstract

Model building and parameter estimation are traditional concepts widely used in chemical, biological, metallurgical, and manufacturing industries. Early modeling methodologies focused on mathematically capturing the process knowledge and domain expertise of the modeler. The models thus developed are termed first principles models (or white‐box models). Over time, computational power became cheaper, and massive amounts of data became available for modeling. This led to the development of cutting edge machine learning models (black‐box models) and artificial intelligence (AI) techniques. Hybrid models (gray‐box models) are a combination of first principles and machine learning models. The development of hybrid models has captured the attention of researchers as this combines the best of both modeling paradigms. Recent attention to this field stems from the interest in explainable AI (XAI), a critical requirement as AI systems become more pervasive. This work aims at identifying and categorizing various hybrid models available in the literature that integrate machine‐learning models with different forms of domain knowledge. Benefits such as enhanced predictive power, extrapolation capabilities, and other advantages of combining the two approaches are summarized. The goal of this article is to consolidate the published corpus in the area of hybrid modeling and develop a comprehensive framework to understand the various techniques presented. This framework can further be used as the foundation to explore rational associations between several models.

Application of Hybrid Neural Models to Bioprocesses: A Systematic Literature Review

“…Narayanan et al (2019) reviewed model-based methods for Industry 4.0 emphasizing the potential of hybrid modeling to fulfill Industry 4.0 challenges [11] . Rajulapati et al reviewed the hybrid modeling field in a systems engineering perspective [12] . There are different ways to combine mechanistic models with machine learning into hybrid structures with particular identification challenges [12] .…”

“…Rajulapati et al reviewed the hybrid modeling field in a systems engineering perspective [12] . There are different ways to combine mechanistic models with machine learning into hybrid structures with particular identification challenges [12] . Serial and/or parallel hybrid structures may be static or dynamic, each of them requiring different training methods.…”

Review of Machine Learning for Hydrodynamics, Transport, and Reactions in Multiphase Flows and Reactors

“…One of the typical examples in the context of unsupervised ML is to boost ML prediction capability through unsupervised contrastive pretraining that leverages small data sets of reactions. 343 It should also be noted that integrating ML with first-principles models 344 will play an increasingly important role in diverse multiphase research areas.…”

A General Hybrid Modeling Framework for Systems Biology Applications: Combining Mechanistic Knowledge with Deep Neural Networks under the SBML Standard

“…Schubert et al [4] presented the first industrial application of hybrid modeling (material balance equations combined with neural networks) to a Baker's yeast process. Since the early 1990s, hybrid model structure definition, parameter identification and model-based process control have been extensively covered (e.g., [5][6][7][8][9][10]). Hybrid models were applied to a wide array of microbial, animal cells, mixed microbial and enzyme processes in different industries, such as wastewater treatment, clean energy, biopolymers and biopharmaceutical manufacturing (Agharafeie et al [11]).…”