Optimized succinct data structures for massive data

Abstract

SUMMARYSuccinct data structures provide the same functionality as their corresponding traditional data structure in compact space. We improve on functions rank and select, which are the basic building blocks of FM‐indexes and other succinct data structures. First, we present a cache‐optimal, uncompressed bitvector representation that outperforms all existing approaches. Next, we improve, in both space and time, on a recent result by Navarro and Providel on compressed bitvectors. Last, we show techniques to perform rank and select on 64‐bit words that are up to three times faster than existing methods. In our experimental evaluation, we first show how our improvements affect cache and runtime performance of both operations on data sets larger than commonly used in the evaluation of succinct data structures. Our experiments show that our improvements to these basic operations significantly improve the runtime performance and compression effectiveness of FM‐indexes on small and large data sets. To our knowledge, our improvements result in FM‐indexes that are either smaller or faster than all current state of the art implementations. Copyright © 2013 John Wiley & Sons, Ltd.

Improved and extended locating functionality on compressed suffix arrays

“…The CSAs of sdsl can be parameterized with the described traditional sampling method, which uses a bitmap B to mark the sampled suffixes. It has recently been shown [11] that this sampling strategy, when B is represented as sd vector [27], gives better time/space tradeoffs than a strategy that does not use B but samples every SA[i] with i ≡ 0 mod s. In our first experiment, we compare the traditional sampling using SA s , SA s by just ·n/s samples plus a bitmap B of length n/s to mark those samples in SA s . We opted for = 1/8, so that every SA −1 value can be retrieved in at most 8 steps, and B is represented as an uncompressed bitmap (bit vector).…”

2018 Proceedings of the Twentieth Workshop on Algorithm Engineering and Experiments (ALENEX)

“…This is fastest (and simultaneously smallest) FM-index we are aware of. The implementation is part of the SDSL library [9], which contains state-of-the-art implementations of compressed data structures.…”

Parallel Lightweight Wavelet Tree, Suffix Array and FM-Index Construction

“…Additionally, there are different practical implementations of rank and select structures [GP14,Vig08,Cla96]. Especially the numbers of cache misses play an important role for the performance of these structures on modern machines.…”