Database compression techniques for performance optimization

Aghav, Sushila

doi:10.1109/iccet.2010.5485951

Cited by 12 publications

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to their ability to accommodate many data formats and their support for a straightforward key-value abstraction, they are actively adopted for various services, including social graph analysis, AI/ML services, and distributed databases [10]. Key-value stores utilize a range of compression techniques [11][12][13], including Snappy [14], Zstandard (Zstd) [15], and LZ4 [16]. These approaches will be discussed further in Table 1.…”

Section: Introductionmentioning

confidence: 99%

Requirements and Trade-Offs of Compression Techniques in Key–Value Stores: A Survey

Jaranilla,

Choi

2023

Electronics

View full text Add to dashboard Cite

The prevalence of big data has caused a notable surge in both the diversity and magnitude of data. Consequently, this has prompted the emergence and advancement of two distinct technologies: unstructured data management and data volume reduction. Key–value stores, such as Google’s LevelDB and Meta’s RocksDB, have emerged as a popular solution for managing unstructured data due to their ability to handle diverse data types with a simple key–value abstraction. Simultaneously, a multitude of data management tools have actively adopted compression techniques, such as Snappy and Zstd, to effectively reduce data volume. The objective of this study is to explore how these two technologies influence each other. For this purpose, we first examine a classification of compression techniques and discuss their strength and weakness, especially those adopted by modern key–value stores. We also investigate the internal structures and operations, such as batch writing and compaction, in order to grasp the characteristics of key–value stores. Then, we quantitatively evaluate the compression ratio and performance using RocksDB under diverse compression techniques, block sizes, value sizes, and workloads. Our evaluation shows that compression not only saves storage space but also decreases compaction overhead. It also reveals that compression techniques have their inherent trade-offs, meaning that some provide a better compression ratio, while others yield better compression performance. Based on our evaluation, a number of potential avenues for further research have been identified. These include the exploration of a compression-aware compaction mechanism, selective compression, and revisiting compression granularity.

show abstract

Section: Introductionmentioning

confidence: 99%