FPGA-based Compaction Engine for Accelerating LSM-tree Key-Value Stores

Sun, Xuan; Yu, Jinghuan; Zhou, Zimeng

doi:10.1109/icde48307.2020.00113

Cited by 21 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reducing the CPU overheads for LSM-tree [45] and transfiguring LSM-tree into in-memory database [17] were considered as well. Recent studies [19,20] sought aid from FPGA for hardware supports to accelerate compactions. Comparatively, the idea of DeLSM is more portable and applicable.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, a KV pair would be factually written again whenever it is involved in a further compaction, causing severe write amplification. Researchers have looked into how to reduce the performance penalty of LSM-tree [6,[8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]. Some of them focused on emerging storage devices [6,7,9,15,18,21], while others sought the aid from specific hardware [19,20].…”

Section: Introductionmentioning

confidence: 99%

“…Researchers have looked into how to reduce the performance penalty of LSM-tree [6,[8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]. Some of them focused on emerging storage devices [6,7,9,15,18,21], while others sought the aid from specific hardware [19,20]. In this paper, however, we envisage a solution to revolutionize LSM-tree, which shall be 1) simplistic but efficient, and 2) generic and software-only, for the ease of development and portability.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Accelerating LSM-Tree with the Dentry Management of File System

Hu,

Wang

2021

Preprint

View full text Add to dashboard Cite

The log-structured merge tree (LSM-tree) gains wide popularity in building key-value (KV) stores. It employs logs to back up arriving KV pairs and maintains a few on-disk levels with exponentially increasing capacity limits, resembling a tiered tree-like structure. A level comprises SST files, each of which holds a sequence of sorted KV pairs. From time to time, LSM-tree redeploys KV pairs from a full level to the lower level by compaction, which merge-sorts and moves KV pairs among SST files, thereby incurring substantial disk I/Os.In this paper, we revisit the design of LSM-tree and find that organizing multiple KV pairs in an SST file entails the heavyweight redeployment of actual KV pairs in a compaction. Accordingly we revolutionize the organization of KV pairs by transforming an SST file of KV pairs to an SST directory, in which each KV pair makes into an independent KV file with the key and value as filename and main file contents, respectively. Moving KV pairs in a compaction converts to transferring directory entries (dentrys), which causes concretely fewer disk I/Os. This is the essence of our design named DeLSM. We build a prototype of DeLSM on LevelDB and evaluation results show that it significantly outperforms the state-of-the-art LSM-tree variants in different dimensions.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Accelerating LSM-Tree with the Dentry Management of File System

Hu,

Wang

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Zhang et al [21] and Sun et al [22] proposed acceleration solutions for LSM-based keyvalue stores by offloading compaction tasks to the FPGA. They perform the compaction task, which is considered the main cause of performance degradation, in the dedicated FPGA, so that the host resources are concentrated on normal operations.…”

Section: Near-storage Data Processingmentioning

confidence: 99%

“…They perform the compaction task, which is considered the main cause of performance degradation, in the dedicated FPGA, so that the host resources are concentrated on normal operations. Zhang et al [21] modified Alibaba's proprietary LSM-based key-value store, X-Engine, whereas Sun et al [22] modified LevelDB to verify the effectiveness of their schemes. Ajdari et al proposed FIDR [23] and CIDR [24]: storage systems that provide data reduction functionality, such as deduplication and compression.…”

Section: Near-storage Data Processingmentioning

confidence: 99%

Understanding the Performance Characteristics of Computational Storage Drives: A Case Study with SmartSSD

2021

View full text Add to dashboard Cite

The emerging computational storage drives (CSDs) provide new opportunities by moving data computation closer to the storage. Performing computation within storage drives enables data pre/post-processing without expensive data transfers. Moreover, large amounts of data can be processed in parallel thanks to the nature of the field-programmable gate array (FPGA) included in CSDs. In a CSD, there are several implementation techniques that support parallel processing, each of which provides a different degree of parallelism. However, without sufficient understanding of the parallel processing techniques of CSD, it can lead to overhead due to misuse rather than benefiting from task offloading. Thus, to exploit the best performance of CSDs, it is important to properly adjust the degree of parallelism of each implementation technique. In this paper, we focus on the study of the differences in CSD performance according to various combinations of parallel processing techniques. To investigate the performance differences, we implement and offload the data verification algorithm to the CSD and analyze the performance and resource utilization. The experimental results show that implementing the data verification algorithm with a sufficient understanding of CSD’s parallel processing techniques can improve the performance by up to 20 times. Moreover, even with the same degree of parallelism, the performance can differ by 59% depending on the combination of implementation techniques. These results imply that proper orchestration of different implementation techniques leads to better performance and efficient resource utilization.

show abstract