The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. Electronic supplementary materialThe online version of this article (doi:10.1186/s13321-017-0232-0) contains supplementary material, which is available to authorized users.

The rapid growth of structural information for G-protein-coupled receptors (GPCRs) has led to a greater understanding of their structure, function, selectivity, and ligand binding. Although novel ligands have been identified using methods such as virtual screening, computationally driven lead optimization has been possible only in isolated cases because of challenges associated with predicting binding free energies for related compounds. Here, we provide a systematic characterization of the performance of free-energy perturbation (FEP) calculations to predict relative binding free energies of congeneric ligands binding to GPCR targets using a consistent protocol and no adjustable parameters. Using the FEP+ package, first we validated the protocol, which includes a full lipid bilayer and explicit solvent, by predicting the binding affinity for a total of 45 different ligands across four different GPCRs (adenosine A2AAR, β1 adrenergic, CXCR4 chemokine, and δ opioid receptors). Comparison with experimental binding affinity measurements revealed a highly predictive ranking correlation (average spearman ρ = 0.55) and low root-mean-square error (0.80 kcal/mol). Next, we applied FEP+ in a prospective project, where we predicted the affinity of novel, potent adenosine A2A receptor (A2AR) antagonists. Four novel compounds were synthesized and tested in a radioligand displacement assay, yielding affinity values in the nanomolar range. The affinity of two out of the four novel ligands (plus three previously reported compounds) was correctly predicted (within 1 kcal/mol), including one compound with approximately a tenfold increase in affinity compared to the starting compound. Detailed analyses of the simulations underlying the predictions provided insights into the structural basis for the two cases where the affinity was overpredicted. Taken together, these results establish a protocol for systematically applying FEP+ to GPCRs and provide guidelines for identifying potent molecules in drug discovery lead optimization projects.

Proteochemometric (PCM) modelling is a computational method to model the bioactivity of multiple ligands against multiple related protein targets simultaneously.