Kinetics-Constrained Neural Ordinary Differential Equations: Artificial Neural Network Models tailored for Small Data to boost Kinetic Model Development

Abstract

Artificial neural networks (ANNs) are powerful tools for solving a wide range of tasks in fundamental and applied science. However, training and building reliable ANN models requires a lot of data which so far hinders their wider application in kinetic modelling where typically only small (experimental) datasets are available. In the present work we propose a method to design ANN models for kinetic modelling that can be trained even with small data sets as are typically available. The key idea is to constrain the architecture of the ANN models by integrating kinetic and thermodynamic knowledge leading to what we call Kinetics-Constrained Neural Ordinary Differential Equations (KCNODE). The feasibility and effectiveness of the approach is first demonstrated in a numerical experiment using the catalytic hydrogenation of CO2 to methane as example. Next, we demonstrate the approach for real experimental data of a more complex reaction, the hydrogenation of CO2 to higher hydrocarbons (CO2-FT). Finally, the ANN trained for CO2-FT is used to derive an improved mechanistic model for the reverse water gas shift reaction which is a key reaction in the CO2-FT reaction network. This last step exemplifies how the opportunity to obtain reliable ANN models from small data opens new ways to approach kinetic model development.

A neural network with embedded stoichiometry and thermodynamics for learning kinetics from reactor data

“…In this work, we show that the new neural network with embedded stoichiometry and thermodynamics, due to the provided background knowledge, greatly simplifies learning kinetic models from reactor data. In parallel to this work, a similar approach using neural ODEs constrained by kinetic background knowledge was followed by Fedorov et al [52].…”

“…Such problems are avoided by our neural network because the thermodynamics layer ensures that the source term of any species becomes positive as its concentration approaches zero (see equation 5), thus preventing negative concentrations. In parallel to this work, Fedorov et al [52] followed a similar approach using thermodynamic and stoichiometric constraints on a neural ODE.…”

Robust Mechanism Discovery with Atom Conserving Chemical Reaction Neural Networks

“…In this work we focus on implementing the atom balance in neural networks. Previously, this has either been done explicitly through a soft constraint in the loss function [4,5], or a post-processing step [6], or indirectly by embedding the stoichiometric matrix into the model's structure [7][8][9][10]. However, in case of mechanism discovery and reduction, the stoichiometric matrix is generally unknown and subject to optimization.…”