A Step Towards Generalisability: Training a Machine Learning Scoring Function for Structure-Based Virtual Screening

Abstract

Over the last few years, many new machine learning-based scoring functions for predicting the binding of small molecules to proteins have been developed. Their objective is to approximate the distribution which takes two molecules as input and outputs the energy of their interaction. This distribution is dependent on the interatomic interactions involved in binding, and only a scoring function that accounts for these interactions can accurately predict binding affinity on unseen molecules. To try to create a method capable of learning these interactions, we built PointVS: a machine learning-based scoring function which achieves state of the art performance even after rigorous filtering of the training set. This filtering is key, as we found that a commonly used benchmark, CASF-16, overestimates the true accuracy of machine learning-based scoring functions when trained using the most commonly used training set. Ranking algorithms using this benchmark rewards memorisation of training data rather than knowledge of the rules of intermolecular binding. We demonstrate that PointVS is able to identify important interactions using attribution, and further, that it can be used to extract important binding pharmacophores from a given protein target when supplied with a number of bound structures. We use this information to perform fragment elaboration, and see improvements in docking scores compared to using structural information from a traditional data-based approach. This not only provides definitive proof that PointVS is learning to identify important binding interactions, but also constitutes the first deep learning-based method for extracting structural information from a target for molecule design.

Exploring The Ability Of Machine Learning-Based Virtual Screening Models To Identify The Functional Groups Responsible For Binding

“…Finally we compared the performance of the fingerprintbased models to a recently proposed Equivariant graph neural network, PointVS. 22 We found that although all models were able to accurately predict binding in the presence of ligand-specific biases, their ability to attribute binding to the correct functional groups was substantially degraded, indicating they were less able to generalise than models which were not susceptible to ligand-specific biases. We also found that the attribution performance of the fingerprint-based models was heavily dependent on the parameters used to define the fingerprint, and that they were less able to identify the most important functional groups compared to the EGNN method, PointVS.…”

“…It must instead learn to identify important interactions from the atomic coordinates and atom types. Scantlebury et al 22 also applied a further distance cutoff, where any receptor atom which was not within 6 Å of any ligand atom was ignored; this reduced the dimensionality of the input graph by ignoring residues which were not part of the binding pocket. As we constrained each synthetic protein to be within a box defined as [x min −5, x max +5]×[y min −5, y max +5]× [z min − 5, z max + 5], where x min was the smallest ligand atom x-coordinate and the other values were defined similarly, the vast majority of synthetic residues would be within 6 Å of at least one ligand atom and so this cutoff should have minimal impact on the performance of PointVS.…”

“…We trained 8 different RF PLEC models, varying the PLEC distance cutoff by 0.5 Å from 2.5 Å to 6 Å. To train PointVS, we used the default hyperparameters outlined by Scantlebury et al 22 .…”

“…As higher scoring molecules are classified as active examples, masking assigns a high level of importance to atoms whose omission drastically reduces the model's confidence that an example has an active label. Whilst Scantlebury et al 22 used an attention mechanism to score the relative importance of different atomic interactions, we used masking for the experiments in this paper as it can be used to generate attributions for any predictive model, allowing a closer comparison between different models.…”

“…Whilst several authors have used attribution 2 techniques on real-world data to uncover important functional groups, 16,21,22 it is often difficult to ascertain the precise contribution of each atom in an experimentally obtained proteinligand complex. Combined with the difficulty in manually curating a large-scale test set, it is currently infeasible to objectively assess the attribution performance of ML algorithms on real-world virtual screening tasks.…”