Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models

Abstract

Graph neural networks (GNN) has been considered as an attractive modelling method for molecular property prediction, and numerous studies have shown that GNN could yield more promising results than traditional descriptor-based methods. In this study, based on 11 public datasets covering various property endpoints, the predictive capacity and computational efficiency of the prediction models developed by eight machine learning (ML) algorithms, including four descriptor-based models (SVM, XGBoost, RF and DNN) and four graph-based models (GCN, GAT, MPNN and Attentive FP), were extensively tested and compared. The results demonstrate that on average the descriptor-based models outperform the graph-based models in terms of prediction accuracy and computational efficiency. SVM generally achieves the best predictions for the regression tasks. Both RF and XGBoost can achieve reliable predictions for the classification tasks, and some of the graph-based models, such as Attentive FP and GCN, can yield outstanding performance for a fraction of larger or multi-task datasets. In terms of computational cost, XGBoost and RF are the two most efficient algorithms and only need a few seconds to train a model even for a large dataset. The model interpretations by the SHAP method can effectively explore the established domain knowledge for the descriptor-based models. Finally, we explored use of these models for virtual screening (VS) towards HIV and demonstrated that different ML algorithms offer diverse VS profiles. All in all, we believe that the off-the-shelf descriptor-based models still can be directly employed to accurately predict various chemical endpoints with excellent computability and interpretability.

Topological Distance-Based Electron Interaction Tensor: A Novel Molecular Structure Representation to Bridge Convolutional Neural Network Studies in Computer Vision to Drug-Like Compound Datasets

“…This was similar to the previous study where increasing the weights within the neural network architecture did not always improve prediction accuracy [20]. Moreover, it is important to search for a model architecture with the minimum number of weights and the highest prediction accuracy, because the use of an excessive number of weights in the model could induce false positives in prediction outcomes [17]. In QSAR modeling, datasets were collected from a wide range of studies in which experimental values were measured using different experimental protocols.…”

“…However, Jian et al experimented with diverse datasets used in deep learning model development studies to compare the prediction accuracy between deep learning, and feature-based ML models. This study showed that the feature-based ML models outperformed the deep learning models in terms of prediction accuracy [17]. Moreover, ML models do not require demanding computation in the training process; therefore, the feature-based ML algorithm is a much more e cient way of developing the model.…”

“…In this study, four datasets were selected for the regression tasks: melting point (MP), water solubility (Esol), octanol/water distribution coe cient (Lipop), and hydration free energy (Freesolv). There are various publicly available datasets for the classi cation problem that are used in deep learning model development studies [9,10,13,17] such as human immunode ciency virus replication inhibition (HIV), human ß-secretase 1 inhibition (BACE), blood-brain barrier penetration (BBBP), toxicity in clinical trials (ClinTox), drug adverse reactions (SIDER), biological targets screened in Tox21 and ToxCast, and PubChem BioAssay data (MUV); however, they were not used in this study because labels in the datasets were seriously imbalanced, whether they were binary, or multiple classi cation tasks.…”

“…It is the largest publicly available, labeled chemical dataset. ESOL, Freesolv, and Lipop were obtained from the study by Jian et al [17]. Because the datasets were already divided into the three given categories by the authors, I used them as such.…”

“…The MP dataset was obtained from the work ofIgor V. Tetko et al [22], and others were obtained from Dejun Jian et al [17].…”