deBWT: parallel construction of Burrows–Wheeler Transform for large collection of genomes with de Bruijn-branch encoding

Abstract

Motivation: With the development of high-throughput sequencing, the number of assembled genomes continues to rise. It is critical to well organize and index many assembled genomes to promote future genomics studies. Burrows–Wheeler Transform (BWT) is an important data structure of genome indexing, which has many fundamental applications; however, it is still non-trivial to construct BWT for large collection of genomes, especially for highly similar or repetitive genomes. Moreover, the state-of-the-art approaches cannot well support scalable parallel computing owing to their incremental nature, which is a bottleneck to use modern computers to accelerate BWT construction.Results: We propose de Bruijn branch-based BWT constructor (deBWT), a novel parallel BWT construction approach. DeBWT innovatively represents and organizes the suffixes of input sequence with a novel data structure, de Bruijn branch encoding. This data structure takes the advantage of de Bruijn graph to facilitate the comparison between the suffixes with long common prefix, which breaks the bottleneck of the BWT construction of repetitive genomic sequences. Meanwhile, deBWT also uses the structure of de Bruijn graph for reducing unnecessary comparisons between suffixes. The benchmarking suggests that, deBWT is efficient and scalable to construct BWT for large dataset by parallel computing. It is well-suited to index many genomes, such as a collection of individual human genomes, with multiple-core servers or clusters.Availability and implementation: deBWT is implemented in C language, the source code is available at https://github.com/hitbc/deBWT or https://github.com/DixianZhu/deBWTContact: ydwang@hit.edu.cnSupplementary information: Supplementary data are available at Bioinformatics online.

The design and construction of reference pangenome graphs with minigraph

“…A straightforward way to represent a pangenome is to store unaligned genomes in a full-text index that compresses redundancies in sequences identical between individuals [8][9][10]. We may retrieve individual genomes from the index, inspect the k-mer spectrum and test the presence of k-mers using standard techniques.…”

FindeR: Accelerating FM-Index-Based Exact Pattern Matching in Genomic Sequences through ReRAM Technology

“…In the configuration stage, programmers can easily specify the basic FindeR parameters, e.g., the BWT and FM-Index files, alphabet, FM-Index bucket width, bank number and RHU number, in the configuration file. We assume the BWT construction of the reference genomes and read pools are done in the cloud [56], [57], so that we can perform trillions of backward searches on them during all steps of genome analysis. At the beginning of compiling, the files of the BWT and FM-Index are copied into ReRAM chips and the other parameters are written into the SMC on the NVDIMM.…”

Survey and Taxonomy of Lossless Graph Compression and Space-Efficient Graph Representations

“…Schemes Based on Burrows-Wheeler Transform Various works incorporate the Burrows-Wheeler Transform for more space efficiency [37,196,304,390] 4.3.2 Grammar-and Text-Related Works. Peshkin [367] uses the notions from both graph grammars and graph compression to understand the structure of DNA and simultaneously be able to represent it compactly.…”