Lightweight merging of compressed indices based on BWT variants

Egidi, Lavinia; Manzini, Giovanni

doi:10.1016/j.tcs.2019.11.001

Cited by 4 publications

(4 citation statements)

References 39 publications

(74 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this variant, the output consists of the last characters of the lexicographically sorted cyclic rotations of all factors of the Lyndon Factorization [16] of s. This variant has been recently shown in [3] to satisfy 1, but being based on the cyclic rotations of the Lyndon factors properties 2 and 3 have not been studied. Note that the related variant Extended BWT [46], which takes as input a collection of strings, does satisfy property 3 for the problem of circular pattern search [24,29,40,39,10,11].…”

Section: Known Bwt Variantsmentioning

confidence: 99%

A New Class of String Transformations for Compressed Text Indexing

Giancarlo¹,

Manzini²,

Restivo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Introduced about thirty years ago in the field of Data Compression, the Burrows-Wheeler Transform (BWT) is a string transformation that, besides being a booster of the performance of memoryless compressors, plays a fundamental role in the design of efficient self-indexing compressed data structures. Finding other string transformations with the same remarkable properties of BWT has been a challenge for many researchers for a long time. Among the known BWT variants, the only one that has been recently shown to be a valid alternative to BWT is the Alternating BWT (ABWT), another invertible string transformation introduced about ten years ago in connection with a generalization of Lyndon words. In this paper, we introduce a whole class of new string transformations, called local orderings-based transformations, which have all the "myriad virtues" of BWT. We show that this new family is a special case of a much larger class of transformations, based on context adaptive alphabet orderings, that includes BWT and ABWT. Although all transformations support pattern search, we show that, in the general case, the transformations within our larger class may take quadratic time for inversion and pattern search. As a further result, we show that the local orderings-based transformations can be used for the construction of the recently introduced r-index, which makes them suitable also for highly repetitive collections. In this context, we consider the problem of finding, for a given string, the BWT variant that minimizes the number of runs in the transformed string, and we provide an algorithm solving this problem in linear time.

show abstract

Section: Known Bwt Variantsmentioning

confidence: 99%

A New Class of String Transformations for Compressed Text Indexing

Giancarlo¹,

Manzini²,

Restivo³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Merging two de Bruijn graphs G 0 and G 1 , or other succinct indices [19], amounts to building a new succinct data structure that supports the retrieval of the elements which are in G 0 or in G 1 . Because of the correspondence between succinct data structures and Wheeler automata, the natural generalization of the problem of merging succinct indices is the problem of computing a Wheeler automaton recognizing the union language L = L 0 ∪ L 1 , given the two Wheeler automata A 0 = (V 0 , E 0 , F 0 , s 0 , Σ) and…”

Section: Merging Wheeler Graphs Via 2-satmentioning

confidence: 99%

“…Notice that the working space of our algorithm is always less than the space of the resulting succinct de Bruijn graph. Our new merging algorithm is based on a mixed LSD/MSD Radix Sort algorithm which is inspired by the lightweight BWT merging introduced by Holt and McMillan [28,29] and later improved in [18,19]. In addition to its small working space, our algorithm has the remarkable feature that it can compute as a by-product, with no additional cost, the LCS (Longest Common Suffix) between the node labels in Bowe et al's representation, thus making it possible to construct succinct Variable Order de Bruijn graph [10], a feature not shared by any other merging algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…In the second part of this paper, we address the issue of generalizing the results on de Bruijn graphs, and some previous results on succinct data structure merging [19,29], to Wheeler graphs. The notion of Wheeler graph has been recently introduced in [26] to provide a unifying view of a large family of compressed data structures loosely based on the BWT [12] or one of its variants.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Space-Efficient Merging of Succinct de Bruijn Graphs

Egidi

Louza

Manzini

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We propose a new algorithm for merging succinct representations of de Bruijn graphs introduced in [Bowe et al. WABI 2012]. Our algorithm is based on the lightweight BWT merging approach by Holt and McMillan [Bionformatics 2014, ACM-BCB 2014]. Our algorithm has the same asymptotic cost of the state of the art tool for the same problem presented by Muggli et al. [bioRxiv 2017, Bioinformatics 2019], but it uses less than half of its working space. A novel important feature of our algorithm, not found in any of the existing tools, is that it can compute the Variable Order succinct representation of the union graph within the same asymptotic time/space bounds.

show abstract