Document listing on repetitive collections with guaranteed performance

Navarro, Gonzalo

doi:10.1016/j.tcs.2018.11.022

Cited by 13 publications

(16 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Space Count time Navarro [89,Thm. 6] O(z log(n/z)) O(m log n + m log 2+ (n/z)) Navarro [89,Thm. 5] O(g) O(m 2 + m log 2+ g) Mäkinen et al [80,Thm.…”

Section: Indexmentioning

confidence: 99%

Fully Functional Suffix Trees and Optimal Text Searching in BWT-Runs Bounded Space

Gagie

Navarro

Prezza

2020

J. ACM

Self Cite

128

142

View full text Add to dashboard Cite

Indexing highly repetitive texts -such as genomic databases, software repositories and versioned text collections -has become an important problem since the turn of the millennium. A relevant compressibility measure for repetitive texts is r, the number of runs in their Burrows-Wheeler Transforms (BWTs). One of the earliest indexes for repetitive collections, the Run-Length FM-index, used O(r) space and was able to efficiently count the number of occurrences of a pattern of length m in the text (in loglogarithmic time per pattern symbol, with current techniques). However, it was unable to locate the positions of those occurrences efficiently within a space bounded in terms of r. Since then, a number of other indexes with space bounded by other measures of repetitivenessthe number of phrases in the Lempel-Ziv parse, the size of the smallest grammar generating (only) the text, the size of the smallest automaton recognizing the text factors -have been proposed for efficiently locating, but not directly counting, the occurrences of a pattern. In this paper we close this long-standing problem, showing how to extend the Run-Length FM-index so that it can locate the occ occurrences efficiently within O(r) space (in loglogarithmic time each), and reaching optimal time, O(m + occ), within O(r log log w (σ + n/r)) space, for a text of length n over an alphabet of size σ on a RAM machine with words of w = Ω(log n) bits. Within that space, our index can also count in optimal time, O(m). Multiplying the space by O(w/ log σ), we support count and locate in O( m log(σ)/w ) and O( m log(σ)/w + occ) time, which is optimal in the packed setting and had not been obtained before in compressed space. We also describe a structure using O(r log(n/r)) space that replaces the text and extracts any text substring of length in almost-optimal time O(log(n/r) + log(σ)/w). Within that space, we similarly provide direct access to suffix array, inverse suffix array, and longest common prefix array cells, and extend these capabilities to full suffix tree functionality, typically in O(log(n/r)) time per operation. Our experiments show that our O(r)-space index outperforms the space-competitive alternatives by 1-2 orders of magnitude.

show abstract

“…Space Count time Navarro [89,Thm. 6] O(z log(n/z)) O(m log n + m log 2+ (n/z)) Navarro [89,Thm. 5] O(g) O(m 2 + m log 2+ g) Mäkinen et al [80,Thm.…”

Section: Indexmentioning

confidence: 99%

Fully Functional Suffix Trees and Optimal Text Searching in BWT-Runs Bounded Space

Gagie

Navarro

Prezza

2020

J. ACM

Self Cite

128

142

View full text Add to dashboard Cite

show abstract

“…(2) We obtain, in space O(γ log(n/γ)), an index that counts the occurrences of P in T in time O(m + log 2+ (γ log(n/γ))), which outperforms the previous result [Navarro 2019] both in time and space.…”

Section: Introductionmentioning

confidence: 81%

Optimal-Time Dictionary-Compressed Indexes

Christiansen¹,

Ettienne²,

Kociumaka³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

We describe the first self-indexes able to count and locate pattern occurrences in optimal time within a space bounded by the size of the most popular dictionary compressors. To achieve this result we combine several recent findings, including string attractors -new combinatorial objects encompassing most known compressibility measures for highly repetitive texts -, and grammars based on locally-consistent parsing.More in detail, let γ be the size of the smallest attractor for a text T of length n. The measure γ is an (asymptotic) lower bound to the size of dictionary compressors based on Lempel-Ziv, context-free grammars, and many others. The smallest known text representations in terms of attractors use space O(γ log(n/γ)), and our lightest indexes work within the same asymptotic space. Let > 0 be a suitably small constant fixed at construction time, m be the pattern length, and occ be the number of its text occurrences. Our index counts pattern occurrences in O(m + log 2+ n) time, and locates them in O(m + (occ + 1) log n) time. These times already outperform those of most dictionary-compressed indexes, while obtaining the least asymptotic space for any index searching within O((m+occ) polylog n) time. Further, by increasing the space to O(γ log(n/γ) log n), we reduce the locating time to the optimal O(m+occ), and within O(γ log(n/γ) log n) space we can also count in optimal O(m) time. No dictionary-compressed index had obtained this time before. All our indexes can be constructed in O(n) space and O(n log n) expected time.As a byproduct of independent interest, we show how to build, in O(n) expected time and without knowing the size γ of the smallest attractor (which is NP-hard to find), a run-length context-free grammar of size O(γ log(n/γ)) generating (only) T . As a result, our indexes can be built without knowing γ.

show abstract

“…A related simpler model of compression is relative compression [50,51] (see also [5,12,15,33,36,37,38,39]), where we explicitly store a store a single reference string and compress a collection of strings as substrings of the reference string. A similar compression model is also proposed in [26,40,42,43]. The relative compression model compresses efficiently if each string is the result of applying a small number of edits to the base string.…”

Section: Previous Workmentioning

confidence: 99%

Random Access in Persistent Strings and Segment Selection

Bille¹,

Gørtz²

2020

Preprint

View full text Add to dashboard Cite

We consider compact representations of collections of similar strings that support random access queries. The collection of strings is given by a rooted tree where edges are labeled by an edit operation (inserting, deleting, or replacing a character) and a node represents the string obtained by applying the sequence of edit operations on the path from the root to the node. The goal is to compactly represent the entire collection while supporting fast random access to any part of a string in the collection. This problem captures natural scenarios such as representing the past history of a edited document or representing highly-repetitive collections. Given a tree with n nodes, we show how to represent the corresponding collection in O(n) space and optimal O(log n/ log log n) query time. This improves the previous time-space trade-offs for the problem. To obtain our results, we introduce new techniques and ideas, including a reduction to a new geometric line segment selection together with an efficient solution.

show abstract

Document listing on repetitive collections with guaranteed performance

Cited by 13 publications

References 52 publications

Fully Functional Suffix Trees and Optimal Text Searching in BWT-Runs Bounded Space

Fully Functional Suffix Trees and Optimal Text Searching in BWT-Runs Bounded Space

Optimal-Time Dictionary-Compressed Indexes

Random Access in Persistent Strings and Segment Selection

Contact Info

Product

Resources

About