Lossless indexing with counting de Bruijn graphs

Abstract

Sequencing data are rapidly accumulating in public repositories. Making this resource accessible for interactive analysis at scale requires efficient approaches for its storage and indexing. There have recently been remarkable advances in building compressed representations of annotated (or colored) de Bruijn graphs for efficiently indexing k-mer sets. However, approaches for representing quantitative attributes such as gene expression or genome positions in a general manner have remained underexplored. In this work, we propose counting de Bruijn graphs, a notion generalizing annotated de Bruijn graphs by supplementing each node–label relation with one or many attributes (e.g., a k-mer count or its positions). Counting de Bruijn graphs index k-mer abundances from 2652 human RNA-seq samples in over eightfold smaller representations compared with state-of-the-art bioinformatics tools and is faster to construct and query. Furthermore, counting de Bruijn graphs with positional annotations losslessly represent entire reads in indexes on average 27% smaller than the input compressed with gzip for human Illumina RNA-seq and 57% smaller for Pacific Biosciences (PacBio) HiFi sequencing of viral samples. A complete searchable index of all viral PacBio SMRT reads from NCBI's Sequence Read Archive (SRA) (152,884 samples, 875 Gbp) comprises only 178 GB. Finally, on the full RefSeq collection, we generate a lossless and fully queryable index that is 4.6-fold smaller than the MegaBLAST index. The techniques proposed in this work naturally complement existing methods and tools using de Bruijn graphs, and significantly broaden their applicability: from indexing k-mer counts and genome positions to implementing novel sequence alignment algorithms on top of highly compressed graph-based sequence indexes.

Label-guided seed-chain-extend alignment on annotated De Bruijn graphs

“…For compatibility with minigraph, we use GetBlunted to derive a variation graph from the De Bruijn graph [17]. As an additional evaluation, we align the test reads to the original virus reference genomes using the TCG-Aligner [33] (the basis for MetaGraph-LA) to determine reference values for alignment accuracy during our experiments. See Supplementary Table A1 for statistics about these graphs.…”

“…A chain is a series of anchors that appear in the correct order with respect to the query such that each anchor can reach the subsequent anchor in the chain via graph traversal. A chain is scored more favorably if it contains more anchors and is penalized if the distances between the anchors in the query differ from their corresponding graph traversal distances [39,33,2].…”

“…Our algorithm for computing label-consistent alignments, called MetaGraph-LA, is a seed-and-extend algorithm incorporating methods from MetaGraph-Align [32] and the TCG-Aligner [33]. Given a seed length l ≤ k and a query sequence Q, we extract each l-mer from Q and find all graph nodes with matching l-length suffixes and fetch their corresponding labels.…”

“…After seed anchoring, we extend each anchor forward and backward using the same strategy as the TCG-Aligner [33], albeit along label-consistent walks. The alignments terminate after reaching a query end, or after satisfying one of the termination criteria described by Karasikov, Mustafa, et al [33,32].…”

Aligning Distant Sequences to Graphs using Long Seed Sketches

“…Since the time complexity of optimal sequence-to-graph alignment grows linearly with the number of edges in the graph [20,16], many approaches instead follow an approximate seed-and-extend strategy [2], which operates in four main steps: i) seed extraction , which in its simplest form involves finding all substrings with a certain length, ii) seed anchoring , finding matching nodes in the graph, iii) seed filtration , often involving clustering [9,37] or co-linear chaining [25,1,32,8] of seeds, and iv) seed extension , involving performing semi-global pairwise sequence alignment forwards and backwards from each anchored seed [28]. We will review the usage of exact seeds utilized in tools such as vg[15] and G raph A ligner [37] and discuss their limitations in a high mutation-rate setting.…”