Fast Compaction Algorithms for NoSQL Databases

Ghosh, Mainak; Gupta, Indranil; Gupta, Shalmoli; Kumar, Nirman

doi:10.1109/icdcs.2015.53

Cited by 6 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several methods based on a log-structured file system have been developed [3], [17], [18], [20], [22], [23], [24], [31], [32], [34]. The LWC tree [10] offers metadata-based compaction without separating keys from values.…”

Section: Related Workmentioning

confidence: 99%

CaseDB: Lightweight Key-Value Store for Edge Computing Environment

Tulkinbekov

Kim

2020

IEEE Access

View full text Add to dashboard Cite

Key-value stores based on a log-structured merge (LSM) tree have emerged in big data systems because of their scalability and reliability. An LSM-tree offers a multilevel data structure with a simple interface. However, it performs file rewrites at the disk level, which causes write amplification. This study is concerned with this problem in relation to an embedded board environment, which can be used in edge computing. Addressing the major problems associated with an LSM-tree, we propose a new key-value store named CaseDB, which aggressively separates keys and bloom filters on the non-volatile memory express (NVMe) drive and stores the values on the SSD. Our solution reduces the I/O cost and enhances the overall performance in a cost-efficient manner. CaseDB employs a memory component, CBuffer, to avoid small write operations, and a delayed value compaction technique that guarantees the sorted order for both keys and values. CaseDB also utilizes deduction-based data deduplication to prevent space amplification in the values layer. The experiments show that CaseDB outperforms LevelDB and WiscKey 5.7 and 1.8 times, respectively, with respect to data writes, and additionally improves the read performance by 1.5 times. CaseDB also avoids the space amplification of WiscKey.

show abstract

Section: Related Workmentioning

confidence: 99%

CaseDB: Lightweight Key-Value Store for Edge Computing Environment

Tulkinbekov

Kim

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We remark that a problem similar to PLT training cost has been studied in the database literature. In particular, the problem of finding an optimal binary tree is proven to be NP-hard in (Ghosh et al, 2015). Note that the result of (Ghosh et al, 2015) does not imply hardness of the PLT training cost problem.…”

Section: Hardness Of Training Cost Minimizationmentioning

confidence: 99%

“…The problem of optimizing the training cost is closely related to the binary merging problem in databases (Ghosh et al, 2015). The hardness result in (Ghosh et al, 2015), however, does not generalize to our setting as it is limited to binary trees only. Nevertheless, our approximation result is partly based on the results from (Ghosh et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

“…The hardness result in (Ghosh et al, 2015), however, does not generalize to our setting as it is limited to binary trees only. Nevertheless, our approximation result is partly based on the results from (Ghosh et al, 2015). The training cost we use is similar to the one considered in , but the authors there consider a specific class of shallow trees.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the computational complexity of the probabilistic label tree algorithms

Busa-Fekete¹,

Dembczyński²,

Golovnev³

et al. 2019

Preprint

View full text Add to dashboard Cite

Label tree-based algorithms are widely used to tackle multi-class and multi-label problems with a large number of labels. We focus on a particular subclass of these algorithms that use probabilistic classifiers in the tree nodes. Examples of such algorithms are hierarchical softmax (HSM), designed for multi-class classification, and probabilistic label trees (PLTs) that generalize HSM to multi-label problems. If the tree structure is given, learning of PLT can be solved with provable regret guaranties (Wydmuch et al., 2018). However, to find a tree structure that results in a PLT with a low training and prediction computational costs as well as low statistical error seems to be a very challenging problem, not well-understood yet.In this paper, we address the problem of finding a tree structure that has low computational cost. First, we show that finding a tree with optimal training cost is NP-complete, nevertheless there are some tractable special cases with either perfect approximation or exact solution that can be obtained in linear time in terms of the number of labels m. For the general case, we obtain O(log m) approximation in linear time too. Moreover, we prove an upper bound on the expected prediction cost expressed in terms of the expected training cost. We also show that under additional assumptions the prediction cost of a PLT is O(log m).

show abstract

“…Optimized Multi-BRWT Construction. By analogy to the NoSQL table compaction problem [9], it can be shown that Multi-BRWT constrained on the space of binary trees with the uncompressed bit vector representation as the underlying structure for storing the index vectors is NP-hard. Thus, we propose a two-step approach for finding a good Multi-BRWT structure (see Figure 2).…”

Section: Multiary Topology-optimized Brwtsmentioning

confidence: 99%

Sparse Binary Relation Representations for Genome Graph Annotation

Karasikov

Mustafa

Joudaki

et al. 2018

Preprint

View full text Add to dashboard Cite

High-throughput DNA sequencing data is accumulating in public repositories, and efficient approaches for storing and indexing such data are in high demand. In recent research, several graph data structures have been proposed to represent large sets of sequencing data and to allow for efficient querying of sequences. In particular, the concept of labeled de Bruijn graphs has been explored by several groups. While there has been good progress towards representing the sequence graph in small space, methods for storing a set of labels on top of such graphs are still not sufficiently explored. It is also currently not clear how characteristics of the input data, such as the sparsity and correlations of labels, can help to inform the choice of method to compress the graph labeling. In this work, we present a new compression approach, Multi-BRWT, which is adaptive to different kinds of input data. We show an up to 29% improvement in compression performance over the basic BRWT method, and up to a 68% improvement over the current state-of-the-art for de Bruijn graph label compression. To put our results into perspective, we present a systematic analysis of five different state-of-the-art annotation compression schemes, evaluate key metrics on both artificial and real-world data and discuss how different data characteristics influence the compression performance. We show that the improvements of our new method can be robustly reproduced for different representative real-world datasets.

show abstract

Fast Compaction Algorithms for NoSQL Databases

Cited by 6 publications

References 13 publications

CaseDB: Lightweight Key-Value Store for Edge Computing Environment

CaseDB: Lightweight Key-Value Store for Edge Computing Environment

On the computational complexity of the probabilistic label tree algorithms

Sparse Binary Relation Representations for Genome Graph Annotation

Contact Info

Product

Resources

About