Transparent Online Storage Compression at the Block-Level

Abstract

In this work, we examine how transparent block-level compression in the I/O path can improve both the space efficiency and performance of online storage. We present ZBD, a block-layer driver that transparently compresses and decompresses data as they flow between the file-system and storage devices. Our system provides support for variable-size blocks, metadata caching, and persistence, as well as block allocation and cleanup. ZBD targets maintaining high performance, by mitigating compression and decompression overheads that can have a significant impact on performance by leveraging modern multicore CPUs through explicit work scheduling. We present two case-studies for compression. First, we examine how our approach can be used to increase the capacity of SSD-based caches, thus increasing their cost-effectiveness. Then, we examine how ZBD can improve the efficiency of online disk-based storage systems.We evaluate our approach in the Linux kernel on a commodity server with multicore CPUs, using PostMark, SPECsfs2008, TPC-C, and TPC-H. Preliminary results show that transparent online block-level compression is a viable option for improving effective storage capacity, it can improve I/O performance up to 80% by reducing I/O traffic and seek distance, and has a negative impact on performance, up to 34%, only when single-thread I/O latency is critical. In particular, for SSD-based caching, our results indicate that, in line with current technology trends, compressed caching trades off CPU utilization for performance and enhances SSD efficiency as a storage cache up to 99%.

Reducing Storage in Large-Scale Photo Sharing Services using Recompression

“…The use of compression for improving the efficiency of storage systems dates back several decades. These involve techniques for achieving compression in file systems such as [21], [22], [23], [24], in databases [25], [26] or for unstructured inputs [27]. ROMP is inspired by this line of work, especially the idea of "online compression" [22], [21], [23], but focuses on a relatively new class of large-scale storage systems specifically designed for photos, whose requirements and workloads are different than those considered by prior works.…”

“…These involve techniques for achieving compression in file systems such as [21], [22], [23], [24], in databases [25], [26] or for unstructured inputs [27]. ROMP is inspired by this line of work, especially the idea of "online compression" [22], [21], [23], but focuses on a relatively new class of large-scale storage systems specifically designed for photos, whose requirements and workloads are different than those considered by prior works. Unlike file or database compression systems, ROMP must compress already compressed objects, leveraging the observation that large coding tables can provide compact storage across the entire storage system (which file and database compression systems can leverage, but, to our knowledge, do not).…”