3D Convolutional Neural Networks and a CrossDocked Dataset for Structure-Based Drug Design

Abstract

One of the main challenges in drug discovery is predicting protein-ligand binding affinity. Recently, machine learning approaches have made substantial progress on this task. However, current methods of model evaluation are overly optimistic in measuring generalization to new targets, and there does not exist a standard dataset of sufficient size to compare performance between models. We present a new dataset for structure-based machine learning, the CrossDocked2020 set, with 22.5 million poses of ligands docked into multiple similar binding pockets across the Protein Data Bank and perform a comprehensive evaluation of grid-based convolutional neural network models on this dataset. We also demonstrate how the partitioning of the training data and test data can impact the results of models trained with the PDBbind dataset, how performance improves by adding more, lower-quality training data, and how training with docked poses imparts pose sensitivity to the predicted affinity of a complex. Our best performing model, an ensemble of 5 densely connected convolutional newtworks, achieves a root mean squared error of 1.42 and Pearson R of 0.612 on the affinity prediction task, an AUC of 0.956 at binding pose classification, and a 68.4% accuracy at pose selection on the CrossDocked2020 set. By providing data splits for clustered cross-validation and the raw data for the CrossDocked2020 set, we establish the first standardized dataset for training machine learning models to recognize ligands in non-cognate target structures while also greatly expanding the number of poses available for training. In order to facilitate community adoption of this dataset for benchmarking protein-ligand binding affinity prediction, we provide our models, weights, and the CrossDocked2020 set at https://github.com/gnina/models.

NeuralDock: Rapid and conformation-agnostic docking of small molecules

“…We found agreement of MedusaDock E without VDWR and NeuralDock E without VDWR with experimental pK, with similar correlations for both tools (r=-0.48 for both, p<0.0001), and no statistically significant difference in correlations with 2-way ANCOVA (F=1.27, p=0.26), (Figure 3b). The correlation of MedusaDock E without VDWR with experimental pK (r=−0.48) is comparable to that of AutoDock Vina scoring (r=0.41) as reported in Francoeur et al 23 , with MedusaDock performing better, likely due to our extensive sampling and computational effort for each protein-ligand pair. NeuralDock predicts experimental pK better than MedusaDock on the validation set (Figures 1b, 1c, 3b), however experimental validation of the pKs is needed to confirm that this result holds for SOD1 and other proteins.…”

“…Due to the inclusion of protein-small molecule crystal structures, K DEEP is biased and has limited generalizability to proteins not bound to small molecules. This issue of bias has been discussed in Francoeur et al 23 and arises from self-docking, in which the neural network is provided with a low conformational energy crystal structure as input, and therefore does not perform conformational sampling. Cang et al 26 use convolutional networks with manually constructed ligand features based on ligand topology, but provide no forward validation with docking tools; instead, only binding affinity is predicted, increasing the risk of overfitting and self-docking bias.…”

“…In stark contrast to work such as in Francoeur et al 23 , we do not explicitly train our neural network to predict the energy of a specific conformation. Rather, we train our network in a conformation-invariant manner by withholding conformational information in the inputs and by predicting population parameters like minimum energy and binding affinity.…”

“…Deep neural networks for predicting binding affinities have been successful, however there are drawbacks to specific approaches [23][24][25][26] . For example, Francoeur et al 23 proposed to approximate force fields using neural network-based scoring while still relying on extensive conformational sampling, hence retaining the major computational bottleneck of virtual docking. Gentile et al 24 used neural networks to aid in chemical screening, but the predictions were nonspecific: information about the protein pocket was not used in the screening 24 .…”

Development of a graph convolutional neural network model for efficient prediction of protein-ligand binding affinities

“…Francoeur et al . [ 8 ] have found that data augmentation with low quality re-docking data can improve model performance. Despite the caution from the authors that the improvement might have resulted from the expanded data volume, their report shows that data augmentation with structural information improve the predicting power of a model.…”