Building an Efficient Put-Intensive Key-Value Store with Skip-Tree

Yue, Yinliang; He, Bingsheng; Li, Yuzhe; Wang, Weiping

doi:10.1109/tpds.2016.2609912

Cited by 36 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other proposed improvements, such as the skip-tree [81], TRIAD [16], and the VT-tree [64], propose several new ideas to improve the write performance of LSM-trees. However, in addition to extra overhead on query performance and space utilization, these optimizations may bring non-trivial implementation complexity to real systems.…”

Section: Discussion Of Overall Trade-offsmentioning

confidence: 99%

“…In contrast, dCompaction [53] offers no builtin support for workload balancing. It is not clear how skewed SSTable groups would impact the performance of these struc- ♣ LWC-tree [78,79] ♣ PebblesDB [58] ♣ dCompaction [53] ♣ Zhang et al [82] ♣ SifrDB [50] ♣ Skip-tree [81] ♣ TRIAD [16] ♣ VT-tree [64] ♣ Zhang et al [84] ♣ Ahmad et al [8] ♣ LSbM-tree [68,69] ♣ bLSM [61] ♣ FloDB [15] ♣ Accordion [19] ♣ cLSM [34] ♣ FD-tree [44] ♣ FD+tree [71] ♣ MaSM [13] ♣ WiscKey [46] ♣ HashKV [20] ♣ Kreon [54] ♣ NoveLSM [39] ♣ LDS [49] ♣ LOCS [74] ♣ NoFTL-KV [73] ♣ LHAM [51] ♣ LSM-trie [76] ♣ SlimDB [59] ♣ Mathieu et al [48] ♣ Lim et al [45] ♣ Monkey [25,26] ♣ Dostoevsky [24] ♣ Thonangi and Yang [70] ♣ ElasticBF [83] ♣ Mutant [80] ♣ LSII [75] ♣ Kim et al [42] ♣ Filter [11] ♣ Qader et al [57] ♣ Diff-Index [66] ♣ DELI [67], ♣ Luo and Carey …”

Section: Tieringmentioning

confidence: 99%

“…The skip-tree [81] proposes a merge skipping idea to improve write performance. The observation is that each entry must be merged from level 0 down to the largest level.…”

Section: Merge Skippingmentioning

confidence: 99%

See 2 more Smart Citations

LSM-based storage techniques: a survey

2019

View full text Add to dashboard Cite

Recently, the Log-Structured Merge-tree (LSMtree) has been widely adopted for use in the storage layer of modern NoSQL systems. Because of this, there have been a large number of research efforts, from both the database community and the operating systems community, that try to improve various aspects of LSM-trees. In this paper, we provide a survey of recent research efforts on LSM-trees so that readers can learn the state-of-the-art in LSM-based storage techniques. We provide a general taxonomy to classify the literature of LSM-trees, survey the efforts in detail, and discuss their strengths and trade-offs. We further survey several representative LSM-based open-source NoSQL systems and discuss some potential future research directions resulting from the survey.

show abstract

Section: Discussion Of Overall Trade-offsmentioning

confidence: 99%

Section: Tieringmentioning

confidence: 99%

See 1 more Smart Citation

LSM-based storage techniques: a survey

2019

View full text Add to dashboard Cite

show abstract

“…LWC-store [51] decouples data and metadata management in compaction by merging and sorting only the metadata in SSTables. SkipStore [52] pushes KV pairs across non-adjacent levels to reduce the number of levels traversed during compaction. TRIAD [3] combines different techniques to reduce compaction overhead, and addresses data skewness by keeping hot data in memory while flushing only cold data into disk (note that our write cache also buffers recently written KV pairs and allows them to be directly updated in-place).…”

Section: Related Workmentioning

confidence: 99%

“…Note that HashKV incurs random writes due to hashing, yet our implementation can feasibly mitigate the random access overhead ( §3.8) since SSDs have a closer performance gap between random and sequential writes compared to hard disks [28,37]. While HashKV builds on LevelDB by default for the key and metadata management, it can also build on other KV stores that have more efficient LSM-tree designs (e.g., [38,42,44,49,51,52]). To demonstrate, we replace LevelDB with RocksDB [14], HyperLevelDB [13], and PebblesDB [38] and show that HashKV improves their respective performance via KV separation and more efficient value management.…”

mentioning

confidence: 99%

Enabling Efficient Updates in KV Storage via Hashing

Chan

Lee

2019

ACM Trans. Storage

View full text Add to dashboard Cite

Persistent key-value (KV) stores mostly build on the Log-Structured Merge (LSM) tree for high write performance, yet the LSM-tree suffers from the inherently high I/O amplification. KV separation mitigates I/O amplification by storing only keys in the LSM-tree and values in separate storage. However, the current KV separation design remains inefficient under update-intensive workloads due to its high garbage collection (GC) overhead in value storage. We propose HashKV, which aims for high update performance atop KV separation under update-intensive workloads. HashKV uses hash-based data grouping, which deterministically maps values to storage space so as to make both updates and GC efficient. We further relax the restriction of such deterministic mappings via simple but useful design extensions. We extensively evaluate various design aspects of HashKV. We show that HashKV achieves 4.6× update throughput and 53.4% less write traffic compared to the current KV separation design. In addition, we demonstrate that we can integrate the design of HashKV with state-of-the-art KV stores and improve their respective performance. IntroductionPersistent key-value (KV) stores are an integral part of modern large-scale storage infrastructures for storing massive structured data (e.g., [4,6,11,22]). While many real-world KV storage workloads are read-intensive (e.g., the Get-Update request ratio can reach 30× in Facebook's Memcached workloads [2]), update-intensive workloads are also dominant in many storage scenarios, including online transaction processing [47] and enterprise servers [21]. Field studies show that the amount of write requests becomes more significant in modern enterprise workloads. For example, Yahoo! reports that its low-latency workloads increasingly move from reads to writes [42]; Baidu reports that the read-write request ratio of a cloud storage workload is 2. 78× [22]; Microsoft reports that read-write traffic ratio of a 3-month OneDrive workload is 2.3× [7].Modern KV stores optimize the performance of writes (including inserts and updates) using the Log-Structured Merge (LSM) tree [35]. Its idea is to transform updates into sequential writes through a logstructured (append-only) design [40], while supporting efficient queries including individual key lookups and range scans. In a nutshell, the LSM-tree buffers written KV pairs and flushes them into a multi-level tree, in which each node is a fixed-size file containing sorted KV pairs and their metadata. It stores the recently written KV pairs at higher tree levels, and merges them with lower tree levels via compaction. The LSM-tree design not only improves write performance by avoiding small random updates (which are also harmful to the endurance of solid-state drives (SSDs) [1,33]), but also improves range scan performance by keeping sorted KV pairs in each node.

show abstract