Evaluation ofin silicoalgorithms for use with ACMG/AMP clinical variant interpretation guidelines

Abstract

BackgroundThe American College of Medical Genetics and American College of Pathologists (ACMG/AMP) variant classification guidelines for clinical reporting are widely used in diagnostic laboratories for variant interpretation. The ACMG/AMP guidelines recommend complete concordance of predictions among all in silico algorithms used without specifying the number or types of algorithms. The subjective nature of this recommendation contributes to discordance of variant classification among clinical laboratories and prevents definitive classification of variants.ResultsUsing 14,819 benign or pathogenic missense variants from the ClinVar database, we compared performance of 25 algorithms across datasets differing in distinct biological and technical variables. There was wide variability in concordance among different combinations of algorithms with particularly low concordance for benign variants. We also identify a previously unreported source of error in variant interpretation where in silico predictions are opposite to the evidence provided by other sources. We identified recently developed algorithms with high predictive power and robust to variables like disease mechanism, gene constraint and mode of inheritance, although poorer performing algorithms are more frequently used based on review of the clinical genetics literature (2011-2017).ConclusionsOur analyses identify algorithms with high performance characteristics independent of underlying disease mechanisms. We describe combinations of algorithms with increased concordance that should improve in silico algorithm usage during assessment of clinically relevant variants using the ACMG/AMP guidelines.

Genetic variant pathogenicity prediction trained using disease-specific clinical sequencing datasets

“…CCR was determined to be the most useful feature for classification, replicating results from the CCR manuscript (Havrilla et al 2019). Consistent with a recent survey of pathogenicity predictor performance using ClinVar variants (Ghosh, Oak, and Plon 2017), we find that VEST outperforms FATHMM for ClinVar variants.…”

Pediatric Cancer Variant Pathogenicity Information Exchange (PeCanPIE): A Cloud-based Platform for Curating and Classifying Germline Variants

“…In comparison with web-based systems (Masica et al 2017) which provide batch annotation of variants based on machine-learning scores (Carter et al 2013, 2009), PeCanPIE provides more granular annotations and individual ACMG-recommended evidence tags to facilitate interpretation of pathogenicity classifications. Via dbNSFP, PeCanPIE also provides access to REVEL (loannidis et al 2016) pathogenicity scores, which fared well in a recent comparison of algorithms for use with ACMG clinical variant interpretation guidelines (Ghosh et al 2017). Lastly, PeCanPIE’s workflow offers advantages over CIVIC’s crowdsourced clinical interpretation of variants (Ta 2017), which relies on completely manual classification and data entry, i.e., VCF upload, annotation, and prioritization are not provided.…”

How I curate: applying American Society of Hematology-Clinical Genome Resource Myeloid Malignancy Variant Curation Expert Panel rules for RUNX1 variant curation for germline predisposition to myeloid malignancies

“…For in-silico evaluation of missense variants, the MM-VCEP recommends using REVEL, a meta-predictor that combines 13 individual tools with high sensitivity and specificity, which has demonstrated the highest performance compared with individual tools or other ensemble methods. [39][40][41] The computational prediction code PP3 is applica-ble to the p.His105Pro variant due to a high REVEL score of 0.953 (MM-VCEP defined >0.75 as the cutoff). The ClinVar submitter (SCV000807773.1) provided us with the patient's clinical data from their laboratory and the proband met at least one of the RUNX1 phenotype criteria (Table 1) which qualified for PS4_supporting.…”