Reply to Comment on ‘Physics-based representations for machine learning properties of chemical reactions’

Abstract

Recently, we published an article in this journal that explored physics-based representations in combination with kernel models for predicting reaction properties (i.e. TS barrier heights). In an anonymous comment on our contribution, the authors argue, amongst other points, that deep learning models relying on atom-mapped reaction SMILES are more appropriate for the same task. This raises the question: are deep learning models sounding the death knell for kernel based models? By studying several datasets that vary in the type of chemical (i.e. high-quality atom-mapping) and structural information (i.e. Cartesian coordinates of reactants and products) contained within, we illustrate that physics-based representations combined with kernel models are competitive with deep learning models. Indeed, in some cases, such as when reaction barriers are sensitive to the geometry, physics-based models represent the only viable candidate. Furthermore, we illustrate that the good performance of deep learning models relies on high-quality atom-mapping, which comes with significant human time-cost and, in some cases, is impossible. As such, both physics-based and graph models offer their own relative benefits to predict reaction barriers of differing datasets.

3DReact: Geometric Deep Learning for Chemical Reactions

“…Models are run in three atom-mapping regimes: (i) with high-quality maps (“True”) derived from the TS structures or heuristic rules; ,,,, (ii) with atom-maps obtained using the open-source RXNMapper (“RXNMapper”); and (iii) without any atom-mapping information at all (“None”). As discussed in recent work, , previously developed graph-based models for reaction property prediction − ,, including ChemProp , reported prediction errors only in the “True” atom-mapping regime. The “RXNMapper” regime is important for cases where the reaction mechanism is not known and atom-mapping using heuristic rules is impossible.…”

Graph to Activation Energy Models Easily Reach Irreducible Errors but Show Limited Transferability

“…Graph-based neural-network models have become notorious in many contexts for overfitting and poor out-of-distribution performance. ,, Although the models trained on RGD1 show excellent testing performance on unseen reactions, this is a large data set, and reactions typically involve a small number of bond changes and conserved mechanisms. This means that even if the testing set involves unseen reactions in terms of reactants or products, it is not expected to necessarily present novel reactivity (e.g., in terms of new types of bonds being broken and formed) that is not seen elsewhere in the training data.…”

“…In this sense, the ideal model would be able to achieve high accuracy based solely on the reactant and product graphs. The negotiation of these trade-offs remains a live issue. ,, …”