Defining a novel k-nearest neighbours approach to assess the applicability domain of a QSAR model for reliable predictions

Abstract

BackgroundWith the growing popularity of using QSAR predictions towards regulatory purposes, such predictive models are now required to be strictly validated, an essential feature of which is to have the model’s Applicability Domain (AD) defined clearly. Although in recent years several different approaches have been proposed to address this goal, no optimal approach to define the model’s AD has yet been recognized.ResultsThis study proposes a novel descriptor-based AD method which accounts for the data distribution and exploits k-Nearest Neighbours (kNN) principle to derive a heuristic decision rule. The proposed method is a three-stage procedure to address several key aspects relevant in judging the reliability of QSAR predictions. Inspired from the adaptive kernel method for probability density function estimation, the first stage of the approach defines a pattern of thresholds corresponding to the various training samples and these thresholds are later used to derive the decision rule. Criterion deciding if a given test sample will be retained within the AD is defined in the second stage of the approach. Finally, the last stage tries reflecting upon the reliability in derived results taking model statistics and prediction error into account.ConclusionsThe proposed approach addressed a novel strategy that integrated the kNN principle to define the AD of QSAR models. Relevant features that characterize the proposed AD approach include: a) adaptability to local density of samples, useful when the underlying multivariate distribution is asymmetric, with wide regions of low data density; b) unlike several kernel density estimators (KDE), effectiveness also in high-dimensional spaces; c) low sensitivity to the smoothing parameter k; and d) versatility to implement various distances measures. The results derived on a case study provided a clear understanding of how the approach works and defines the model’s AD for reliable predictions.

A novel applicability domain technique for mapping predictive reliability across the chemical space of a QSAR: reliability-density neighbourhood

“…This is in line with the theoretical expectation that the training of the QSAR model and the calculation of the AD are two different tasks, as already explained in the “Methods” section.

Fig. 6Comparison of different feature sets used in the dk-NN AD by Sahigara et al [12], applied to the P-gp IV set

…”

In Silico Prediction of Physicochemical Properties of Environmental Chemicals Using Molecular Fingerprints and Machine Learning

“…The average distance of the test chemical from its five nearest neighbors in the training set is compared with a threshold, which is the 95 th percentile of average distance of training chemicals from their five nearest neighbors. 63 …”

Validation and extension of a similarity-based approach for prediction of acute aquatic toxicity towards Daphnia magna

“…According to this approach a molecule is predicted only if its average distance from the first three neighbours is lower than a fixed threshold value; otherwise it is unpredicted and considered out of the applicability domain. In this way, the evaluation of the applicability domain is implicitly defined on a similarity-based approach [28][29][30] and carried out on the fly for each prediction. The introduction of the threshold is meant to identify test molecules that are dissimilar from their nearest neighbours and whose predictions are consequently supposed to be unreliable.…”