BCD deduplication: effective memory compression using partial cache-line deduplication

Park, Sungbo; Kang, Ingab; Moon, Yaebin; Ahn, Jung Ho; Suh, G. Edward

doi:10.1145/3445814.3446722

Cited by 12 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Software memory compression schemes [7,10,11] are often bottlenecked by the computation overhead. Other memory compression schemes [9,[12][13][14][15][16][17][18] are performed by a dedicated processor between the CPU cache and physical memory, which is called hardware memory compression.…”

Section: Memory Compressionmentioning

confidence: 99%

QZRAM: A Transparent Kernel Memory Compression System Design for Memory-Intensive Applications with QAT Accelerator Integration

Gao,

Xu,

Yang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

In recent decades, memory-intensive applications have experienced a boom, e.g., machine learning, natural language processing (NLP), and big data analytics. Such applications often experience out-of-memory (OOM) errors, which cause unexpected processes to exit without warning, resulting in negative effects on a system’s performance and stability. To mitigate OOM errors, many operating systems implement memory compression (e.g., Linux’s ZRAM) to provide flexible and larger memory space. However, these schemes incur two problems: (1) high-compression algorithms consume significant CPU resources, which inevitably degrades application performance; and (2) compromised compression algorithms with low latency and low compression ratios result in insignificant increases in memory space. In this paper, we propose QZRAM, which achieves a high-compression-ratio algorithm without high computing consumption through the integration of QAT (an ASIC accelerator) into ZRAM. To enhance hardware and software collaboration, a page-based parallel write module is introduced to serve as a more efficient request processing flow. More importantly, a QAT offloading module is introduced to asynchronously offload compression to the QAT accelerator, reducing CPU computing resource consumption and addressing two challenges: long CPU idle time and low usage of the QAT unit. The comprehensive evaluation validates that QZRAM can reduce CPU resources by up to 49.2% for the FIO micro-benchmark, increase memory space (1.66×) compared to ZRAM, and alleviate the memory overflow phenomenon of the Redis benchmark.

show abstract

Section: Memory Compressionmentioning

confidence: 99%

QZRAM: A Transparent Kernel Memory Compression System Design for Memory-Intensive Applications with QAT Accelerator Integration

Gao,

Xu,

Yang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

A novel approximate cache block compressor for error-resilient image data

Loloeyan,

Nikmehr,

Rezaei

2024

Computers and Electrical Engineering

View full text Add to dashboard Cite

Research on Data Routing Strategy of Deduplication in Cloud Environment

Zhang

Bian

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The application of data deduplication technology reduces the demand for data storage and improves resource utilization. Compared with limited storage capacity and computing capacity of a single node, cluster data deduplication technology has great advantages. However, the cluster data duplication technology also brings new issues on deduplication rate reduction and load balancing of storage nodes. The application of data routing strategy can well balance the problem of deduplication rate and load balancing. Therefore, this paper proposes a data routing strategy based on distributed Bloom Filter. 1)Superchunk is used as the basic unit of data routing to improve system throughput. According to Broder's theorem, k leastsized fingerprints are selected as the Superchunk features and send to the storage node. The optimal node is selected as the routing node by matching the BloomFilter, and the storage capacity of the node and maintained in the memory of the storage node. 2) Design and implement system prototypes. The specific parameters of all kinds of routing strategies are obtained through experiments, and the routing strategies proposed in this paper are tested. The theoretical analysis and experimental results prove the feasibility of the strategies proposed by this paper. Compared with the other routing strategies, our method improved 3% of the deduplication rate, reduces the communication query overhead by more than 36% and improves the load balancing degree of the storage system.

show abstract

BCD deduplication: effective memory compression using partial cache-line deduplication

Cited by 12 publications

References 12 publications

QZRAM: A Transparent Kernel Memory Compression System Design for Memory-Intensive Applications with QAT Accelerator Integration

QZRAM: A Transparent Kernel Memory Compression System Design for Memory-Intensive Applications with QAT Accelerator Integration

A novel approximate cache block compressor for error-resilient image data

Research on Data Routing Strategy of Deduplication in Cloud Environment

Contact Info

Product

Resources

About