ClinPred: Prediction Tool to Identify Disease-Relevant Nonsynonymous Single-Nucleotide Variants

Abstract

Advances in high-throughput DNA sequencing have revolutionized the discovery of variants in the human genome; however, interpreting the phenotypic effects of those variants is still a challenge. While several computational approaches to predict variant impact are available, their accuracy is limited and further improvement is needed. Here, we introduce ClinPred, an efficient tool for identifying disease-relevant nonsynonymous variants. Our predictor incorporates two machine learning algorithms that use existing pathogenicity scores and, notably, benefits from inclusion of normal population allele frequency from the gnomAD database as an input feature. Another major strength of our approach is the use of ClinVar-a rapidly growing database that allows selection of confidently annotated disease-causing variants-as a training set. Compared to other methods, ClinPred showed superior accuracy for predicting pathogenicity, achieving the highest area under the curve (AUC) score and increasing both the specificity and sensitivity in different test datasets. It also obtained the best performance according to various other metrics. Moreover, ClinPred performance remained robust with respect to disease type (cancer or rare disease) and mechanism (gain or loss of function). Importantly, we observed that adding allele frequency as a predictive feature-as opposed to setting fixed allele frequency cutoffs-boosts the performance of prediction. We provide pre-computed ClinPred scores for all possible human missense variants in the exome to facilitate its use by the community.

Assessing performance of pathogenicity predictors using clinically-relevant variant datasets

“…The two meta-predictors outperformed SIFT and PolyPhen-2 in both datasets. In agreement with tool author benchmarking 14–16 the meta-predictors REVEL, ClinPred and GAVIN were highly proficient at classifying the variants in the open dataset, achieving sensitivities of 0.87, 0.90 and 0.95, and specificities of 0.95, 1.00 and 0.98, respectively. For variants in the clinical dataset, although the sensitivity each tool remained largely constant, the specificity of all tools dropped considerably.…”

“…Since the development of the first in silico prediction tools over a decade ago 5,9 , large-scale experiments such as the ENCODE project 10 have generated huge amounts of functional data, and we now also have access to large-scale databases of clinical and neutral variation 11–13 . These additional sources of data have led to an explosion of new in silico prediction algorithms 14–16 that purport to increase accuracy.…”

“…Studies have previously identified the limitations of applying a strict binary consensus-based approach 20 . In response, multiple groups 14–16 have created meta-predictors; tools which integrate information from a large number of sources into a machine-learning algorithm. These tools thereby adhere to the principle of the consensus-based model suggested by ACMG without the onerous task of determining tool concordance, and reduce discordance when increasingly large numbers of tools are utilised.…”

“…However, the use of standardised datasets may introduce inherent biases into prediction algorithms, resulting in false concordance. Typically, prediction software is trained using machine-learning algorithms, and assessed using variants available from large online public databases 5,6,9,10,14–16,22 such as ExAC/gnomAD, ClinVar 12 , and SwissProt 23 . It has been previously shown that prediction algorithms have variable performance when applied to different datasets 6,22,24,25 , and therefore the use of variant datasets derived from online public databases may not be representative of the performance of tools when applied in a clinical setting.…”

“…Here we evaluate and compare the performance of two traditional in silico pathogenicity prediction tools commonly used for clinical variant interpretation (SIFT 5 and PolyPhen-2 9 ), and three meta-predictors (REVEL 14 , GAVIN 15 and ClinPred 16 ) using a publicly available (‘open’) variant dataset and a clinically-relevant (‘clinical’) variant dataset. We show that the tools’ performance is heavily affected by the test dataset, and that all tools may perform worse than expected when classifying novel missense variants.…”