Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from Different Chemical Spaces

Abstract

In this work, we benchmark a variety of singleand multi-task graph neural network (GNN) models against lower-bar and higher-bar traditional machine learning approaches employing human engineered molecular features. We consider four GNN variants -Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN), and Attentive Fingerprint (AttentiveFP). So far deep learning models have been primarily benchmarked using lower-bar traditional models solely based on fingerprints, while more realistic benchmarks employing fingerprints, whole-molecule descriptors and predictions from other related endpoints (e. g., LogD7.4) appear to be scarce for industrial ADME datasets. In addition to time-split test sets based on Genentech data, this study benefits from the availability of measurements from an external chemical space (Roche data). We identify GAT as a promising approach to implementing deep learning models. While all the deep learning models significantly outperform lower-bar benchmark traditional models solely based on fingerprints, only GATs seem to offer a small but consistent improvement over higher-bar benchmark traditional models. Finally, the accuracy of in vitro assays from different laboratories predicting the same experimental endpoints appears to be comparable with the accuracy of GAT single-task models, suggesting that most of the observed error from the models is a function of the experimental error propagation.

Machine Learning for Fast, Quantum Mechanics-Based Approximation of Drug Lipophilicity

“…Numerous computational approaches for the prediction of log P have been developed. , We provide a brief overview here and point toward more detailed descriptions elsewhere. ,, Computational approaches to log P prediction can be grouped into (i) empirical and (ii) physics-based methods. , Empirical methods (i) include contribution-type approaches (atom- or fragment-based , ), QSAR approaches, and deep learning approaches ,,− trained on experimental data. Contribution-type approaches obtain a log P estimate by dividing molecules into either individual atoms or fragments and summing up their contributions, using correction terms in the latter case .…”

ALipSol: An Attention-Driven Mixture-of-Experts Model for Lipophilicity and Solubility Prediction

“…Although the GSE is easy to use and requires no training, it is based on certain assumptions and performs poorly on large molecules. , Additionally, since the experimental data of log P and p K a is scarce, the calculation of log D or log S w using predicted values usually leads to amplified errors . Another solution is using statistical machine learning (ML) algorithms to train a customized model, such as Gaussian Process Regression, Support Vector Machine, Random Forest, ExtraTrees, eXtreme Gradient Boosting (XGBoost), and Deep Neural Networks. , The descriptor-based models have made remarkable progress in molecular property prediction, but they heavily rely on expert knowledge for the design and selection of informative descriptors, possibly introducing expert bias . Moreover, given the fixed-size vectors as input, the tailored models cannot fully utilize molecular structure information, easily causing overfitting and limited generalization capability to unseen data.…”

“…Recently, graph-based deep learning (DL) methods have attracted a multitude of attention and manifested remarkable effects on drug discovery ranging from molecular property prediction to drug virtual screening . These methods are capable of learning suitable molecular representations directly from chemical graphs in an end-to-end fashion. , Related works in lipophilicity and solubility prediction have verified the superiority of graph representation learning models, including undirected graph recursive neural networks (UGRNN) and a variety of graph neural networks (GNN). ,− …”

“…Thus, naively training p K a together with lipophilicity or solubility tasks in one model is suboptimal and even detrimental. According to the study reported by Broccatelli et al, one potential solution for utilizing the knowledge of task dependencies is designing a network to explicitly model the hierarchy of intertask relationships.…”

Enabling data-limited chemical bioactivity predictions through deep neural network transfer learning

“…Whereas image recognition techniques per se have not been widely adopted for bioactivity prediction problems [8,17], the development of e cient molecular feature extraction methods can roughly be divided into a static, structure-based descriptor method that encodes atom and bond features [18][19][20][21][22] and a dynamic graph neural network (GNN) approach that learns molecular features within the context of the data to be modeled [23][24][25][26]. This latter method represents an e cient end-to-end deep learning method as the learnt, extracted features capture all features important to the modeled data, but may not be applicable for other datasets [27].…”