Online Updates on Data Warehouses via Judicious Use of Solid-State Storage

Athanassoulis, Manos; Chen, Shimin; Ailamaki, Anastasia; Gibbons, Philip B.; Stoica, Radu

doi:10.1145/2699484

Cited by 12 publications

(9 citation statements)

References 40 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, fractured mirrors use both a columnar and a row-wise data representation [70,71]. Other approaches employ a variation of LSM-Trees [60] which implement differential updates [12,13,48,67,78]. Data Blocks [49] support a hybrid block layout and store chunks of data along with metadata that allows for efficient searching, querying, and updating.…”

Section: Related Workmentioning

confidence: 99%

Optimal column layout for hybrid workloads

2019

Self Cite

View full text Add to dashboard Cite

Data-intensive analytical applications need to support both efficient reads and writes. However, what is usually a good data layout for an update-heavy workload, is not well-suited for a read-mostly one and vice versa. Modern analytical data systems rely on columnar layouts and employ delta stores to inject new data and updates. We show that for hybrid workloads we can achieve close to one order of magnitude better performance by tailoring the column layout design to the data and query workload. Our approach navigates the possible design space of the physical layout: it organizes each column's data by determining the number of partitions, their corresponding sizes and ranges, and the amount of buffer space and how it is allocated. We frame these design decisions as an optimization problem that, given workload knowledge and performance requirements, provides an optimal physical layout for the workload at hand. To evaluate this work, we build an in-memory storage engine, Casper, and we show that it outperforms state-of-the-art data layouts of analytical systems for hybrid workloads. Casper delivers up to 2.32× higher throughput for update-intensive workloads and up to 2.14× higher throughput for hybrid workloads. We further show how to make data layout decisions robust to workload variation by carefully selecting the input of the optimization.

show abstract

Section: Related Workmentioning

confidence: 99%

Optimal column layout for hybrid workloads

2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…We continue with another common theme, workload-driven continuous reorganization which can be used either when optimizing for read performance [30,31,32,33,35,38,39,40,41,71] or, with differential updates, for update performance [2,4,5,29,37,42,73,75].…”

Section: Continuous Reorganizationmentioning

confidence: 99%

“…Differential updates have been further studied in the context of updating analytical datasets with the stepped-merge algorithm [42] and the Materialized Sort-Merge (MaSM) [4,5] algorithm. The stepped-merge algorithm stores updates lazily by maintaining updates in memory in sorted runs, and eventually forming a B + -Tree of these updates using an external merge-sort.…”

Section: Continuous Reorganizationmentioning

confidence: 99%

Design Tradeoffs of Data Access Methods

Athanassoulis

Idreos

2016

Proceedings of the 2016 International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Database researchers and practitioners have been building methods to store, access, and update data for more than five decades. Designing access methods has been a constant effort to adapt to the ever-changing underlying hardware and workload requirements. The recent explosion in data system designs-including, in addition to traditional SQL systems, NoSQL, NewSQL, and other relational and nonrelational systems-makes understanding the tradeoffs of designing access methods more important than ever. Access methods are at the core of any new data system. In this tutorial, we survey recent developments in access method design and we place them in the design space where each approach focuses primarily on one or a subset of read performance, update performance, and memory utilization. We discuss how to utilize designs and lessons-learned from past research. In addition, we discuss new ideas on how to build access methods that have tunable behavior, as well as, what is the scenery of open research problems.

show abstract

“…The reason why this cost is high, is that in order to access the k th bit we need to decode all previous bits; there is no way to decode a subset of the bitvector knowing in which position the decoding starts with the existing encoding schemes. We address this performance limitation by adding fence pointers which enable efficient partial decoding close to any arbitrary position 3 . Both encoded and not encoded bitvectors are organized in words.…”

Section: Upbit Operationsmentioning

confidence: 99%

“…The major drawback of bitmaps indexes is offering updates. This is exacerbated by the increasing need of applications to require support for both efficient reads and updates, to deliver both good read performance and data freshness [2,3,5,17,22,30].…”

Section: Introductionmentioning

confidence: 99%

UpBit

Athanassoulis

Yan

Idreos

2016

Proceedings of the 2016 International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Bitmap indexes are widely used in both scientific and commercial databases. They bring fast read performance for specific types of queries, such as equality and selective range queries. A major drawback of bitmap indexes, however, is that supporting updates is particularly costly. Bitmap indexes are kept compressed to minimize storage footprint; as a result, updating a bitmap index requires the expensive step of decoding and then encoding a bitvector. Today, more and more applications need support for both reads and writes, blurring the boundaries between analytical processing and transaction processing. This requires new system designs and access methods that support general updates and, at the same time, offer competitive read performance. In this paper, we propose scalable in-memory Updatable Bitmap indexing (UpBit), which offers efficient updates, without hurting read performance. UpBit relies on two design points. First, in addition to the main bitvector for each domain value, UpBit maintains an update bitvector, to keep track of updated values. Effectively, every update can now be directed to a highly-compressible, easy-toupdate bitvector. While update bitvectors double the amount of uncompressed data, they are sparse, and as a result their compressed size is small. Second, we introduce fence pointers in all update bitvectors which allow for efficient retrieval of a value at an arbitrary position. Using both synthetic and real-life data, we demonstrate that UpBit significantly outperforms state-of-the-art bitmap indexes for workloads that contain both reads and writes. In particular, compared to update-optimized bitmap index designs UpBit is 15 − 29× faster in terms of update time and 2.7× faster in terms of read performance. In addition, compared to read-optimized bitmap index designs UpBit achieves efficient and scalable updates (51 − 115× lower update latency), while allowing for comparable read performance, having up to 8% overhead.

show abstract

Online Updates on Data Warehouses via Judicious Use of Solid-State Storage

Cited by 12 publications

References 40 publications

Optimal column layout for hybrid workloads

Optimal column layout for hybrid workloads

Design Tradeoffs of Data Access Methods

UpBit

Contact Info

Product

Resources

About