UpBit

Abstract

Bitmap indexes are widely used in both scientific and commercial databases. They bring fast read performance for specific types of queries, such as equality and selective range queries. A major drawback of bitmap indexes, however, is that supporting updates is particularly costly. Bitmap indexes are kept compressed to minimize storage footprint; as a result, updating a bitmap index requires the expensive step of decoding and then encoding a bitvector. Today, more and more applications need support for both reads and writes, blurring the boundaries between analytical processing and transaction processing. This requires new system designs and access methods that support general updates and, at the same time, offer competitive read performance. In this paper, we propose scalable in-memory Updatable Bitmap indexing (UpBit), which offers efficient updates, without hurting read performance. UpBit relies on two design points. First, in addition to the main bitvector for each domain value, UpBit maintains an update bitvector, to keep track of updated values. Effectively, every update can now be directed to a highly-compressible, easy-toupdate bitvector. While update bitvectors double the amount of uncompressed data, they are sparse, and as a result their compressed size is small. Second, we introduce fence pointers in all update bitvectors which allow for efficient retrieval of a value at an arbitrary position. Using both synthetic and real-life data, we demonstrate that UpBit significantly outperforms state-of-the-art bitmap indexes for workloads that contain both reads and writes. In particular, compared to update-optimized bitmap index designs UpBit is 15 − 29× faster in terms of update time and 2.7× faster in terms of read performance. In addition, compared to read-optimized bitmap index designs UpBit achieves efficient and scalable updates (51 − 115× lower update latency), while allowing for comparable read performance, having up to 8% overhead.

Tree-Encoded Bitmaps

“…The implementation has the advantages that we can work with multiple iterators in parallel and that the most important operations can be performed in a single instruction. For instance, multiple iterators can be incremented using a single masked shift instruction (_mm512_mask_slli_epi16 4 ) and all iterators can be dereferenced in parallel to retrieve the aforementioned α value; cf., Figure 10.…”

“…5 The naïve and costly way to support random updates is to decompress the bitmap, perform the update on the uncompressed representation, and (re-)compress it again afterwards. Prior work [4] proposed to reduce the update costs by staging updates in an auxiliary differential data structure and to apply these pending updates in batches, rather than one-by-one. Thereby, another compressed bitmap is used as a differential data structure.…”

“…We observed a 30% penalty for TEB and Roaring with 20k pending updates (20% with WAH), irrespective of partitioning. For more general information on the trade-offs involved with differential updates, we refer the reader to UpBit [4].…”

“…Throughout the paper, we already covered the related work regarding bitmap compression techniques [2,4,15,18,22,27,32,33,57,63], except for the HICAMP bitmap [56] which is designed for a special kind of memory system [13]. In the following, we discuss other related work.…”