Large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery

Abstract

Structure–activity relationship modelling is frequently used in the early stage of drug discovery to assess the activity of a compound on one or several targets, and can also be used to assess the interaction of compounds with liability targets. QSAR models have been used for these and related applications over many years, with good success. Conformal prediction is a relatively new QSAR approach that provides information on the certainty of a prediction, and so helps in decision-making. However, it is not always clear how best to make use of this additional information. In this article, we describe a case study that directly compares conformal prediction with traditional QSAR methods for large-scale predictions of target-ligand binding. The ChEMBL database was used to extract a data set comprising data from 550 human protein targets with different bioactivity profiles. For each target, a QSAR model and a conformal predictor were trained and their results compared. The models were then evaluated on new data published since the original models were built to simulate a “real world” application. The comparative study highlights the similarities between the two techniques but also some differences that it is important to bear in mind when the methods are used in practical drug discovery applications. Electronic supplementary material The online version of this article (10.1186/s13321-018-0325-4) contains supplementary material, which is available to authorized users.

Systematic analysis of protein targets associated with adverse events of drugs from clinical trials and post-marketing reports

“…Firstly, since plasma concentrations were only available for a subset of the data, this likely reduced the power of our study. It is also well known that bioactivity datasets are sparse and incomplete, further limiting the power and recall (21). Our analysis did not take into account functional effects, such as agonism or antagonism, as this information is not consistently available from the databases considered here (2,21).…”

“…It is also well known that bioactivity datasets are sparse and incomplete, further limiting the power and recall (21). Our analysis did not take into account functional effects, such as agonism or antagonism, as this information is not consistently available from the databases considered here (2,21). This may have resulted in the masking of associations only associated with certain functional effects or modes of action.…”

“…In case of protein complexes, bioactivities at each of the constituent single proteins listed in the target_components table were used. The activity types included were 'IC50', 'EC50', 'XC50', 'AC50', 'Ki', 'Kd' and 'Potency' for assay of types 'B' (binding) and 'F' (functional), with standard_flag = 1 and confidence scores of 7 (protein complexes) and 9 (single proteins), ensuring the highest level of confidence in target assignment (21). Inactive data was retrieved by extracting records containing any of the following in the activity_comment: 'Not active', 'inactive', 'No inhibition' and allowing standard_flag = 0 for these records.…”

“…Next, we used the in vitro bioactivity data from the ChEMBL database and supplemented this with ligand-based target predictions, which was necessary due to the known sparsity of ChEMBL data ( Fig. 1, steps 4 and 5) (21,22). This step hence provided us with drug-target associations.…”

MAIP: A Prediction Platform for Predicting Blood-Stage Malaria Inhibitors

“…It is usually calculated from the variables used to describe the model training set. Methods such as conformal prediction are furthermore able to quantify the prediction certainty [31][32][33]. Our consensus approaches were developed with the underlying training sets remaining obscured, leaving us with limited options to perform further analysis and in particular it was not possible to directly assess the model's domain of applicability.…”