Transfer Learning for a Foundational Chemistry Model

Abstract

Data-driven chemistry has garnered much interest concurrent with improvements in hardware and the development of new machine learning models. However, a notable bottleneck for data-driven chemistry specifically is the challenge in obtaining sufficiently large, accurate datasets of a desired chemical outcome. Herein, I develop a machine learning framework that makes prediction amid low data: First, a chemical “foundational model” is trained using on a dataset of ~1 million experimental organic crystal structures of organic molecules - a source of big data in the chemistry. A task specific model is then stacked on top on this general model. This approach achieves state-of-the-art performance in a diverse set of tasks – toxicity prediction, yield prediction, and odor prediction. More generally, my work shows that a foundational model approach, which led to step-change in domains such as natural language, can unlock advances in chemistry.

Self-Driving Laboratories for Chemistry and Materials Science

“…DL methods are capable of transfer learning, a technique commonly used in low-data settings, in which the model is pre-trained with more readily available data that provides the model with implicit information about the main task. 220,225,226 By leveraging the libraries of computational results, and historical empirical results, models can be preconditioned with physicochemical information for SDL campaigns that typically start in the low-data regime. Such models can even be used to encode chemical compounds as task-specific descriptors, compressing the chemical information into expressive abstract representations.…”

The Influence of Biofuels Addition on Shock-Induced Ignition and Combustion of Methane–Hydrogen Mixtures

“…It has gained popularity, as it implements a modern approach to describing polycyclic hydrocarbons formation, and can also be supplemented with NO x and soot formation reaction submodules. The performance of the mechanisms on methane-hydrogen mixtures autoignition was also compared with other classical kinetic schemes-GRI-Mech 3.0, which has been the industry standard for the last two decades [27]; FFCM-1, resulting from modern efforts of global constrained optimization within the uncertainties of reaction rate parameters [28]; and Aramco 2.0 [29].…”