Themisto: a scalable coloredk-mer index for sensitive pseudoalignment against hundreds of thousands of bacterial genomes

Abstract

Motivation: Huge data sets containing whole-genome sequences of bacterial strains are now commonplace and represent a rich and important resource for modern genomic epidemiology and metagenomics. In order to efficiently make use of these data sets, efficient indexing data structures - that are both scalable and provide rapid query throughput - are paramount. Results: Here, we present Themisto, a scalable colored k-mer index designed for large collections of microbial reference genomes, that works for both short and long read data. Themisto indexes 179 thousand Salmonella enterica genomes in 9 hours. The resulting index takes 142 gigabytes, and Themisto pseudoaligns reads from a Salmonella enterica isolate sample against the index at a rate of 2 million base pairs per second on 48 threads. In comparison, the best competing tools Metagraph and Bifrost were only able to index 11 thousand genomes in the same time. In pseudoalignment, these other tools were either an order of magnitude slower than Themisto, or used an order of magnitude more memory. Themisto also offers superior pseudoalignment quality, achieving a higher recall than previous methods on Nanopore read sets. Availability and implementation: Themisto is available and documented as a C++ package at https://github.com/algbio/themisto available under the GPLv2 license.

Seamless, rapid and accurate analyses of outbreak genomic data using Split K-mer Analysis (SKA)

“…SKA2 uses exact matching of k-mers, which has been the focus of intense optimisation efforts in bioinformatics, to create SNP alignments without explicitly performing any alignment. This is very similar to the pseudo-alignment approach which has become the dominant form of analysis of RNA-seq data (Bray et al 2016) and for representing population data (Holley and Melsted 2020;Alanko et al 2023;Břinda et al 2023). SKA2 has very similar benefits -speed, ease of use, robustness to structural variation, and reduced reference bias.…”

kmindex and ORA: indexing and real-time user-friendly queries in terabyte-sized complex genomic datasets

“…We evaluated the performances of together with eight state-of-the-art k -mer indexers: [2]; [7]; [18]; [17]; [22]; [13]; [12]; and [3]. The dataset for this benchmark is composed of metagenomic seawater sequencing data from 50 Tara Oceans samples, of 1.4TB of gzipped fastq files.…”

The Backpack Quotient Filter: a dynamic and space-efficient data structure for queryingk-mers with abundance.

“…k -mers are to genomics what words are to natural language: this way we can compare sequences by comparing their words. The number of k -mers existing in two sequences provides a metric to measure the similarity between them, leading to the so called pseudo-alignment [2]. In order to efficiently perform pseudo-alignments between any queried sequence and a dataset, we index its k -mers.…”

“…Due to the scale of databases to index, recognized tools often sacrifice precision for the sake of performance. This can be done through pseudo-alignment as defined in [2], breaking down the queried sequences into k -mers and comparing them against k -mers of the datasets, often organised in “colored de Bruijn graph” representation of as in Bifrost [12] or GGCAT [9]. Here, the graph construction is the main limitation of the methods.…”