Skmer: assembly-free and alignment-free sample identification using genome skims

Sarmashghi, Shahab; Bohmann, Kristine; Gilbert, M. Thomas P.; Bafna, Vineet; Mirarab, Siavash

doi:10.1186/s13059-019-1632-4

Cited by 105 publications

(142 citation statements)

References 82 publications

Supporting

Mentioning

142

Contrasting

Order By: Relevance

“…These approaches rely on assembly construction pipelines (e.g., Jin et al, 2019) to remove contaminants (i.e., to avoid mis-assembly or to filter out mis-assembled contigs). Elsewhere, we have advocated going beyond organelle genomes and using all reads in an assembly-free fashion to increase the resolution of taxonomic identification (Balaban et al, 2019;Sarmashghi et al, 2019). However, this goal has been hampered by the presence of contaminants.…”

Section: Discussionmentioning

confidence: 99%

“…Recently, Sarmashghi et al, 2019 developed a method, Skmer, that accurately computes genomic distance between genome skims by simply analyzing k-mers (short substrings of length k) in both genome skims.…”

Section: Introductionmentioning

confidence: 99%

“…These distances can then be used for taxonomic identification and phylogenetic placement (Balaban et al, 2019) of query genome skims with respect to a set of reference genome skims. Previous results have shown high accuracy and increased resolution compared to barcodes when using genome skims for taxonomic identification (Balaban et al, 2019;Sarmashghi et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the impact of contaminants on the accuracy of genome skimming and the effectiveness of exclusion read filters

Rachtman

Balaban

Bafna

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

The ability to detect the identity of a sample obtained from its environment is a cornerstone of molecular ecological research. Thanks to the falling price of shotgun sequencing, genome skimming, the acquisition of short reads spread across the genome at low coverage, is emerging as an alternative to traditional barcoding. By obtaining far more data across the whole genome, skimming has the promise to increase the precision of sample identification beyond traditional barcoding while keeping the costs manageable.While methods for assembly-free sample identification based on genome skims are now available, little is known about how these methods react to the presence of DNA from organisms other than the target species. In this paper, we show that the accuracy of distances computed between a pair of genome skims based on k-mer similarity can degrade dramatically if the skims include contaminant reads; i.e., any reads originating from other organisms. We establish a theoretical model of the impact of contamination. We then suggest and evaluate a solution to the contamination problem: Query reads in a genome skim against an extensive database of possible contaminants (e.g., all microbial organisms) and filter out any read that matches. We evaluate the effectiveness of this strategy when implemented using Kraken-II, in detailed analyses. Our results show substantial improvements in accuracy as a result of filtering but also point to limitations, including a need for relatively close matches in the contaminant database.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the impact of contaminants on the accuracy of genome skimming and the effectiveness of exclusion read filters

Rachtman

Balaban

Bafna

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Under standard implementations where computing set unions and set intersections has a linear time complexity in the sizes of the sets, this filtering step has a time complexity of O(n 2 L) for pairwise read alignment. Recently Jaccard similarity has been used in a variety of applications such as genome skimming [Denver et al, 2016], and in new methods to compare whole genomes and study taxonomic diversity in the microbiome [Ondov et al, 2016, Sarmashghi et al, 2019.…”

Section: Introductionmentioning

confidence: 99%

Spectral Jaccard Similarity: A new approach to estimating pairwise sequence alignments

Baharav

Kamath

Tse

et al. 2019

Preprint

View full text Add to dashboard Cite

A key step in many genomic analysis pipelines is the identification of regions of similarity between pairs of DNA sequencing reads. This task, known as pairwise sequence alignment, is a heavy computational burden, particularly in the context of third-generation long-read sequencing technologies, which produce noisy reads. This issue is commonly addressed via a two-step approach: first, we filter pairs of reads which are likely to have a large alignment, and then we perform computationally intensive alignment algorithms only on the selected pairs. The Jaccard similarity between the set of k-mers of each read can be shown to be a proxy for the alignment size, and is usually used as the filter. This strategy has the added benefit that the Jaccard similarities don't need to be computed exactly, and can instead be efficiently estimated through the use of min-hashes. This is done by hashing all k-mers of a read and computing the minimum hash value (the min-hash) for each read. For a randomly chosen hash function, the probability that the min-hashes are the same for two distinct reads is precisely their k-mer Jaccard similarity. Hence, one can estimate the Jaccard similarity by computing the fraction of min-hash collisions out of the set of hash functions considered.However, when the k-mer distribution of the reads being considered is significantly non-uniform, Jaccard similarity is no longer a good proxy for the alignment size. In particular, genome-wide GC biases and the presence of common k-mers increase the probability of a min-hash collision, thus biasing the estimate of alignment size provided by the Jaccard similarity. In this work, we introduce a min-hash-based approach for estimating alignment sizes called Spectral Jaccard Similarity which naturally accounts for an uneven k-mer distribution in the reads being compared. The Spectral Jaccard Similarity is computed by considering a min-hash collision matrix (where rows correspond to pairs of reads and columns correspond to different hash functions), removing an offset, and performing a singular value decomposition. The leading left singular vector provides the Spectral Jaccard Similarity for each pair of reads. In addition, we develop an approximation to the Spectral Jaccard Similarity that can be computed with a single matrix-vector product, instead of a full singular value decomposition.We demonstrate improvements in AUC of the Spectral Jaccard Similarity based filters over Jaccard Similarity based filters on 40 datasets of PacBio reads from the NCTC collection. The code is available at https://github.com/TavorB/spectral_jaccard_similarity. * Contributed equally and listed alphabetically

show abstract

“…In Mash [39], the MinHash [6] approach is used to reduce the input sequences to small 'sketches' which can be used to rapidly approximate the Jaccard index. Skmer [47] is a further improvement of this approach. In a previous paper, we proposed another way to infer evolutionary distances between DNA sequences based on the number of word matches between them, and we generalized this to so-called spaced-word matches [36].…”

Section: Introductionmentioning

confidence: 99%

The number of spaced-word matches between twoDNAsequences as a function of the underlying pattern weight

Röhling

Dencker

Morgenstern

2019

Preprint

View full text Add to dashboard Cite

We study the number N k of length-k word matches between pairs of evolutionarily related DNA sequences, as a function of k. We show that the Jukes-Cantor distance between two genome sequences -i.e. the number of substitutions per site that occurred since they evolved from their last common ancestor -can be estimated from the slope of a function F that depends on N k and that is affine-linear within a certain range of k. Integers k min and k max can be calculated depending on the length of the input sequences, such that the slope of F in the relevant range can be estimated from the values F (k min ) and F (k max ). This approach can be generalized to so-called spaced-word matches, where mismatches are allowed at positions specified by a user-defined binary pattern. Based on these theoretical results, we implemented a 1 software program for alignment-free sequence comparison called Slope-SpaM. Test runs on real and simulated sequence data show that Slope-SpaM can accurately estimate phylogenetic distances for distances up to around 0.5 substitutions per position. The statistical stability of our results is improved if spaced words are used instead of contiguous words. Unlike previous alignment-free methods that are based on the number of (spaced) word matches, Slope-SpaM produces accurate results, even if sequences share only local homologies.

show abstract

Skmer: assembly-free and alignment-free sample identification using genome skims

Cited by 105 publications

References 82 publications

On the impact of contaminants on the accuracy of genome skimming and the effectiveness of exclusion read filters

On the impact of contaminants on the accuracy of genome skimming and the effectiveness of exclusion read filters

Spectral Jaccard Similarity: A new approach to estimating pairwise sequence alignments

The number of spaced-word matches between twoDNAsequences as a function of the underlying pattern weight

Contact Info

Product

Resources

About