Comparison of INDEL Calling Tools with Simulation Data and Real Short-Read Data

Li, Donghe; Kim, Wonji; Wang, Long; Yoon, Kyong–Ah; Park, Boyoung; Park, Charny; Kong, Sun‐Young; Hwang, Yongdeuk; Baek, Daehyun; Lee, Eun Sook; Won, Sungho

doi:10.1109/tcbb.2018.2854793

Cited by 5 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, it is reasonable to believe that compared to SNP calls, a greater proportion of the indel calls are incorrect, irrespective of whether reads are trimmed or not. This reflects the technical difficulty in indel calling, which often necessitates dedicated algorithms (see performance reviews [26,27]). In my dataset, the majority of indel calls are mixed ( Fig.…”

Section: Read Trimming Has Slightly Greater Effect On Indel Calling Cmentioning

confidence: 99%

Read trimming has minimal effect on bacterial SNP-calling accuracy

Bush

2020

Microbial Genomics

View full text Add to dashboard Cite

Read alignment is the central step of many analytic pipelines that perform variant calling. To reduce error, it is common practice to pre-process raw sequencing reads to remove low-quality bases and residual adapter contamination, a procedure collectively known as ‘trimming’. Trimming is widely assumed to increase the accuracy of variant calling, although there are relatively few systematic evaluations of its effects and no clear consensus on its efficacy. As sequencing datasets increase both in number and size, it is worthwhile reappraising computational operations of ambiguous benefit, particularly when the scope of many analyses now routinely incorporates thousands of samples, increasing the time and cost required. Using a curated set of 17 Gram-negative bacterial genomes, this study initially evaluated the impact of four read-trimming utilities (Atropos, fastp, Trim Galore and Trimmomatic), each used with a range of stringencies, on the accuracy and completeness of three bacterial SNP-calling pipelines. It was found that read trimming made only small, and statistically insignificant, increases in SNP-calling accuracy even when using the highest-performing pre-processor in this study, fastp. To extend these findings, >6500 publicly archived sequencing datasets from Escherichia coli , Mycobacterium tuberculosis and Staphylococcus aureus were re-analysed using a common analytic pipeline. Of the approximately 125 million SNPs and 1.25 million indels called across all samples, the same bases were called in 98.8 and 91.9 % of cases, respectively, irrespective of whether raw reads or trimmed reads were used. Nevertheless, the proportion of mixed calls (i.e. calls where <100 % of the reads support the variant allele; considered a proxy of false positives) was significantly reduced after trimming, which suggests that while trimming rarely alters the set of variant bases, it can affect the proportion of reads supporting each call. It was concluded that read quality- and adapter-trimming add relatively little value to a SNP-calling pipeline and may only be necessary if small differences in the absolute number of SNP calls, or the false call rate, are critical. Broadly similar conclusions can be drawn about the utility of trimming to an indel-calling pipeline. Read trimming remains routinely performed prior to variant calling likely out of concern that doing otherwise would typically have negative consequences. While historically this may have been the case, the data in this study suggests that read trimming is not always a practical necessity.

show abstract

Section: Read Trimming Has Slightly Greater Effect On Indel Calling Cmentioning

confidence: 99%

Read trimming has minimal effect on bacterial SNP-calling accuracy

Bush

2020

Microbial Genomics

View full text Add to dashboard Cite

show abstract

“…Commercial solutions and/or in-house proprietary tools are also increasingly being used by NGS analysis providers. Comparisons of the performance of different pipelines is limited 5,6 . This makes the evaluation, standardisation and regulation of NGS variant calling performance difficult 7 .…”

Section: Introductionmentioning

confidence: 99%

ICR142 Benchmarker: evaluating, optimising and benchmarking variant calling using the ICR142 NGS validation series

Ruark¹,

Holt²,

Renwick³

et al. 2018

Wellcome Open Res

View full text Add to dashboard Cite

Evaluating, optimising and benchmarking of next generation sequencing (NGS) variant calling performance are essential requirements for clinical, commercial and academic NGS pipelines. Such assessments should be performed in a consistent, transparent and reproducible fashion, using independently, orthogonally generated data. Here we present ICR142 Benchmarker, a tool to generate outputs for assessing variant calling performance using the ICR142 NGS validation series, a dataset of exome sequence data from 142 samples together with Sanger sequence data at 704 sites. ICR142 Benchmarker provides summary and detailed information on the sensitivity, specificity and false detection rates of variant callers. ICR142 Benchmarker also automatically generates a single page report highlighting key performance metrics and how performance compares to widely-used open-source tools. We used ICR142 Benchmarker with VCF files outputted by GATK, OpEx and DeepVariant to create a benchmark for variant calling performance. This evaluation revealed pipeline-specific differences and shared challenges in variant calling, for example in detecting indels in short repeating sequence motifs. We next used ICR142 Benchmarker to perform regression testing with versions 0.5.2 and 0.6.1 of DeepVariant. This showed that v0.6.1 improves variant calling performance, but there was evidence of some minor changes in indel calling behaviour that may benefit from attention in future updates. The data also allowed us to evaluate filters to optimise DeepVariant calling, and we recommend using 30 as the QUAL threshold for base substitution calls when using DeepVariant v0.6.1. Finally, we used ICR142 Benchmarker with VCF files from two commercial variant calling providers to facilitate optimisation of their in-house pipelines and to provide transparent benchmarking of their performance. ICR142 Benchmarker consistently and transparently analyses variant calling performance based on the ICR142 NGS validation series, using the standard VCF input and outputting informative metrics to enable user understanding of pipeline performance. ICR142 Benchmarker is freely available at https://github.com/RahmanTeamDevelopment/ICR142_Benchmarker/releases.

show abstract