Simulation study and comparative evaluation of viral contiguous sequence identification tools

Abstract

Background Viruses, including bacteriophages, are important components of environmental and human associated microbial communities. Viruses can act as extracellular reservoirs of bacterial genes, can mediate microbiome dynamics, and can influence the virulence of clinical pathogens. Various targeted metagenomic analysis techniques detect viral sequences, but these methods often exclude large and genome integrated viruses. In this study, we evaluate and compare the ability of nine state-of-the-art bioinformatic tools, including Vibrant, VirSorter, VirSorter2, VirFinder, DeepVirFinder, MetaPhinder, Kraken 2, Phybrid, and a BLAST search using identified proteins from the Earth Virome Pipeline to identify viral contiguous sequences (contigs) across simulated metagenomes with different read distributions, taxonomic compositions, and complexities. Results Of the tools tested in this study, VirSorter achieved the best F1 score while Vibrant had the highest average F1 score at predicting integrated prophages. Though less balanced in its precision and recall, Kraken2 had the highest average precision by a substantial margin. We introduced the machine learning tool, Phybrid, which demonstrated an improvement in average F1 score over tools such as MetaPhinder. The tool utilizes machine learning with both gene content and nucleotide features. The addition of nucleotide features improves the precision and recall compared to the gene content features alone.Viral identification by all tools was not impacted by underlying read distribution but did improve with contig length. Tool performance was inversely related to taxonomic complexity and varied by the phage host. For instance, Rhizobium and Enterococcus phages were identified consistently by the tools; whereas, Neisseria prophage sequences were commonly missed in this study. Conclusion This study benchmarked the performance of nine state-of-the-art bioinformatic tools to identify viral contigs across different simulation conditions. This study explored the ability of the tools to identify integrated prophage elements traditionally excluded from targeted sequencing approaches. Our comprehensive analysis of viral identification tools to assess their performance in a variety of situations provides valuable insights to viral researchers looking to mine viral elements from publicly available metagenomic data.

Prediction of Prophages and Their Host Ranges in Pathogenic and Commensal Neisseria Species

“…S1B ). Our results are consistent with previous observations that several tools have difficulty predicting Neisseria prophages ( 53 ).…”

The Upper Respiratory Tract Microbiome Network Impacted by SARS-CoV-2

“…In other words, there is still a large difference between the sequences of BatCoV BM48-31 and SARS-CoV-2. The results of Kraken 2 should be robust as long as the difference between the two virus sequences is greater than Kraken 2’s error rate, which seems to be the case from existing applications of Kraken 2 [ 12 ].…”

ViralCC retrieves complete viral genomes and virus-host pairs from metagenomic Hi-C data

“…Long contigs (≥ 3 kbp) assembled from shotgun reads were screened by VirSorter (v1.0.6) 15 with default parameter to identify viral contigs. VirSorter achieved the best F1 score in a recent benchmarking study 72 . Contigs annotated as prophages were removed from the viral sequences (Supplementary Table 10) .…”