Low concordance of multiple variant-calling pipelines: practical implications for exome and genome sequencing

Abstract

BackgroundTo facilitate the clinical implementation of genomic medicine by next-generation sequencing, it will be critically important to obtain accurate and consistent variant calls on personal genomes. Multiple software tools for variant calling are available, but it is unclear how comparable these tools are or what their relative merits in real-world scenarios might be.MethodsWe sequenced 15 exomes from four families using commercial kits (Illumina HiSeq 2000 platform and Agilent SureSelect version 2 capture kit), with approximately 120X mean coverage. We analyzed the raw data using near-default parameters with five different alignment and variant-calling pipelines (SOAP, BWA-GATK, BWA-SNVer, GNUMAP, and BWA-SAMtools). We additionally sequenced a single whole genome using the sequencing and analysis pipeline from Complete Genomics (CG), with 95% of the exome region being covered by 20 or more reads per base. Finally, we validated 919 single-nucleotide variations (SNVs) and 841 insertions and deletions (indels), including similar fractions of GATK-only, SOAP-only, and shared calls, on the MiSeq platform by amplicon sequencing with approximately 5000X mean coverage.ResultsSNV concordance between five Illumina pipelines across all 15 exomes was 57.4%, while 0.5 to 5.1% of variants were called as unique to each pipeline. Indel concordance was only 26.8% between three indel-calling pipelines, even after left-normalizing and intervalizing genomic coordinates by 20 base pairs. There were 11% of CG variants falling within targeted regions in exome sequencing that were not called by any of the Illumina-based exome analysis pipelines. Based on targeted amplicon sequencing on the MiSeq platform, 97.1%, 60.2%, and 99.1% of the GATK-only, SOAP-only and shared SNVs could be validated, but only 54.0%, 44.6%, and 78.1% of the GATK-only, SOAP-only and shared indels could be validated. Additionally, our analysis of two families (one with four individuals and the other with seven), demonstrated additional accuracy gained in variant discovery by having access to genetic data from a multi-generational family.ConclusionsOur results suggest that more caution should be exercised in genomic medicine settings when analyzing individual genomes, including interpreting positive and negative findings with scrutiny, especially for indels. We advocate for renewed collection and sequencing of multi-generational families to increase the overall accuracy of whole genomes.

Integrating human sequence data sets provides a resource of benchmark SNP and indel genotype calls

“…Recent publications have demonstrated hundreds of thousands of differences between variant calls from different whole human genome sequencing methods or different bioinformatics methods [5][6][7][8][9][10][11] . To understand these differences, we describe a high-confidence set of genome-wide genotype calls that can be used as a benchmark.…”

Towards a European consensus for reporting incidental findings during clinical NGS testing

“…The process of identifying mutations in NGS data can broadly be divided into three stages; generation of primary data performed by the sequencer, secondary data which includes derived DNA sequence and alignment of reads, and tertiary interpretation data, including the identification of variants and annotation. Two milestones in data analysis are the primary data (from which all results can be regenerated) and the tertiary interpreted variant files, which can be considered an end product of data analysis and are highly dependent on the steps used during data analysis (for instance, there is a low concordance between several commonly used bioinformatics pipelines for variant calling 33 ). Even the commonly used BAM file 34 does not represent primary sequence data, but is the result of aligning sequence reads to a specific reference genome.…”

Next Generation Sequencing and Its Clinical Applications: The Growing Pains

“…Our results also indicate that using independent library preparation replicates is an effective way to identify false positive calls [5]. Recently, O'Rawe et al showed that NGS analysis of the same data set using different variant caller pipelines often resulted in low concordance [7]. Even though restricting ones focus to only the shared variant calls from multiple data analysis pipelines may be an effective way to eliminate some false positives, this approach will not be able to eliminate certain artifacts as effectively as the triplicate approach.…”