From Theory to Practice: Plug and Play with Succinct Data Structures

Gog, Simon; Beller, Timo; Moffat, Alistair; Petri, Matthias

doi:10.1007/978-3-319-07959-2_28

Cited by 334 publications

(279 citation statements)

References 20 publications

Supporting

Mentioning

278

Contrasting

Order By: Relevance

“…The system is written using ≈ 4000 lines of C++11 code with the help of the sdsl library [4]. We use gcc 4.9.2 running on Ubuntu 15.04 in our experiments, with all optimizations enabled.…”

Section: Methodsmentioning

confidence: 99%

Access Time Tradeoffs in Archive Compression

Petri

Moffat²,

Nagesh³

et al. 2015

Information Retrieval Technology

Self Cite

View full text Add to dashboard Cite

Abstract. Web archives, query and proxy logs, and so on, can all be very large and highly repetitive; and are accessed only sporadically and partially, rather than continually and holistically. This type of data is ideal for compression-based archiving, provided that random-access to small fragments of the original data can be achieved without needing to decompress everything. The recent RLZ (relative Lempel Ziv) compression approach uses a semi-static model extracted from the text to be compressed, together with a greedy factorization of the whole text encoded using static integer codes. Here we demonstrate more precisely than before the scenarios in which RLZ excels. We contrast RLZ with alternatives based on block-based adaptive methods, including approaches that "prime" the encoding for each block, and measure a range of implementation options using both hard-disk (HDD) and solid-state disk (SSD) drives. For HDD, the dominant factor affecting access speed is the compression rate achieved, even when this involves larger dictionaries and larger blocks. When the data is on SSD the same effects are present, but not as markedly, and more complex trade-offs apply.

show abstract

Section: Methodsmentioning

confidence: 99%

Access Time Tradeoffs in Archive Compression

Petri

Moffat²,

Nagesh³

et al. 2015

Information Retrieval Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both Bonsai implementations used the sdsl-lite library [21]. The original Bonsai data structure mainly comprises three sdsl containers: firstly, the int vector<>, which uses a fixed number of bits for each entry, is used for the Q array (also in m-Bonsai).…”

Section: Methodsmentioning

confidence: 99%

Improved Practical Compact Dynamic Tries

Poyias

Raman

2015

String Processing and Information Retrieval

View full text Add to dashboard Cite

Abstract. We consider the problem of implementing a dynamic trie with an emphasis on good practical performance. For a trie with n nodes with an alphabet of size σ, the information-theoretic lower bound is n log σ + O(n) bits. The Bonsai data structure [1] supports trie operations in O(1) expected time (based on assumptions about the behaviour of hash functions). While its practical speed performance is excellent, its space usage of (1 + )n(log σ + O(log log n)) bits, where is any constant > 0, is not asymptotically optimal. We propose an alternative, m-Bonsai, that uses (1 + )n(log σ + O(1)) bits in expectation, and supports operations in O(1) expected time (again based on assumptions about the behaviour of hash functions). We give a heuristic implementation of mBonsai which uses considerably less memory and is slightly faster than the original Bonsai.

show abstract

“…For select 1 , we use the standard "sample and scan" approach [1] used by most select implementations including [10,15,29]. We choose a sampling parameter s and logically divide the raw bit-string into select blocks, where the i-th select block begins at the position of the is-th 1, and scan this select block to answer select 1 (j) queries for j = is + 1, .…”

Section: Overviewmentioning

confidence: 99%

“…We implemented OD and P S using the compressed bit-vector implementations RRR and sdarray from the sdsl-lite library [29] and Okanohara's code [31] repectively, which have low redundancy on dense and sparse bit-strings respectively.…”

Section: Implementation Based On Sectionmentioning

confidence: 99%

“…The default implementation of rank on a codeword concatenates all phrases into a bit-string similar to Corollary 3.1 and stores it in a bit-vector supporting rank [29], together with two words per codeword to allow rank on an individual phrase to be reduced to rank on the bit-vector. select on each phrase is done by explicitly storing the positions of the 1s in the phrase in an array.…”

Section: Implementation Of Codeword Operationsmentioning

confidence: 99%

See 1 more Smart Citation

Compressed Bit Vectors Based on Variable-to-Fixed Encodings

Jo¹,

Joannou²,

Okanohara³

et al. 2014

2014 Data Compression Conference

View full text Add to dashboard Cite

We consider practical implementations of compressed bitvectors, which support rank and select operations on a given bit-string, while storing the bit-string in compressed form. Our approach relies on variable-to-fixed encodings of the bit-string, an approach that has not yet been considered systematically for practical encodings of bitvectors. We show that this approach leads to fast practical implementations with low redundancy (i.e., the space used by the bitvector in addition to the compressed representation of the bit-string), and is a flexible and promising solution to the problem of supporting rank and select on moderately compressible bit-strings, such as those encountered in real-world applications.

show abstract

From Theory to Practice: Plug and Play with Succinct Data Structures

Cited by 334 publications

References 20 publications

Access Time Tradeoffs in Archive Compression

Access Time Tradeoffs in Archive Compression

Improved Practical Compact Dynamic Tries

Compressed Bit Vectors Based on Variable-to-Fixed Encodings

Contact Info

Product

Resources

About