SolvBERT for solvation free energy and solubility prediction: a demonstration of an NLP model for predicting the properties of molecular complexes

Abstract

Deep learning models based on NLP, mainly the Transformer family, have been successfully applied to solve many chemistry-related problems, but their applications are mostly limited to chemical reactions. Meanwhile, solvation is an important concept in physical and organic chemistry, describing the interaction of solutes and solvents. This interaction leads to a solvation complex, a molecular complex similar to a reactant-reagent complex. In this study, we introduced the SolvBERT model, which reads the solute and solvents through the SMILES representation of the solvation complex. SolvBERT is pretrained in an unsupervised learning fashion using a large database of computational solvation free energies. The pretrained model can be used to predict the experimental solvation free energy or solubility, depending on the fine-tuning database. To the best of our knowledge, this multi-task prediction capability has not been observed in previously developed graph-based models for predicting the properties of molecular complexes. Furthermore, the performance of our SolvBERT in predicting solvation free energy is comparable to the state-of-the-art graph-based model DMPNN, mainly due to the clustering feature of the pretraining phase of the model, as demonstrated by the TMAP visualization algorithm.

Designing solvent systems in chemical processes using self-evolving solubility databases and graph neural networks

“…[36][37][38] For more rapid and accurate solubility predictions, various predictive models have been actively developed by analyzing quantitative structure-property relationship (QSPR) 34,[39][40][41][42][43][44] or adopting machine learning (ML) techniques. 34,42,[45][46][47][48][49][50][51][52][53][54][55][56] Particularly, current advanced ML models used graph neural networks (GNNs) combined with interaction layers 47,53,57 recurrent neural networks with attention layers, 45 and natural language processing-based transformers. 54,58 These models achieved accuracies close to experimental uncertainties.…”

“…34,42,[45][46][47][48][49][50][51][52][53][54][55][56] Particularly, current advanced ML models used graph neural networks (GNNs) combined with interaction layers 47,53,57 recurrent neural networks with attention layers, 45 and natural language processing-based transformers. 54,58 These models achieved accuracies close to experimental uncertainties. Furthermore, the development of ML models has been expanded to the prediction of solubility limits at different temperatures, 52 solvation enthalpy, LSER, and solute parameters, 51 and generative models for designing molecules having optimal aqueous solubility.…”