LZ78 Compression in Low Main Memory Space

Arroyuelo, Diego; Cánovas, Rodrigo; Navarro, Gonzalo; Raman, Rajeev

doi:10.1007/978-3-319-67428-5_4

Cited by 6 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also proposed an efficient hash-based trie representation in a similar way to m-Bonsai, which is referred to as FK-hash. 3 Although FK-hash uses O(t log σ + t log t) bits of space, its update algorithm is simple and practically fast. However, we are not aware of any space-efficient approach using them as keyword dictionaries.…”

Section: Theoretical Discussionmentioning

confidence: 99%

“…In both variants, we maintain a hash table H of size m with the load factor α = n/m ≤ 1 to store n nodes. In Section 4.3, we propose a linear-time growing algorithm based on Arroyuelo's approach [3]. Finally, in Section 4.4, we propose NLM data structures designed for PBT and CBT.…”

Section: Representation Based On M-bonsaimentioning

confidence: 99%

“…However, because selecting all children of a node is performed by checking getchild for all possible characters in B, the relocation based on a top-down traversal needs O(nσ) expected time and is therefore only for tiny alphabets practical. Here we describe a bottom-up approach that is based on the approach by Arroyuelo et al[3]. This approach, called growing algorithm, runs in O(n) expected time.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Dynamic Path-Decomposed Tries

Kanda

Köppl

Tabei

et al. 2019

Preprint

View full text Add to dashboard Cite

A keyword dictionary is an associative array whose keys are strings. Recent applications handling massive keyword dictionaries in main memory have a need for a space-efficient implementation. When limited to static applications, there are a number of highly-compressed keyword dictionaries based on the advancements of practical succinct data structures. However, as most succinct data structures are only efficient in the static case, it is still difficult to implement a keyword dictionary that is space efficient and dynamic. In this article, we propose such a keyword dictionary. Our main idea is to embrace the path decomposition technique, which was proposed for constructing cache-friendly tries. To store the path-decomposed trie in small memory, we design data structures based on recent compact hash trie representations. Exhaustive experiments on real-world datasets reveal that our dynamic keyword dictionary needs up to 68% less space than the existing smallest ones.

show abstract

Section: Theoretical Discussionmentioning

confidence: 99%

Section: Representation Based On M-bonsaimentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Path-Decomposed Tries

Kanda

Köppl

Tabei

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…We wrote our code in C++17 and compiled it with gcc version 9.2.1 via the compile flags -O3 -march=native -DNDEBUG. We measured the number of allocated bytes with tudostats, 6 which overrides the standard memory allocation (new and malloc) to additionally monitor the maximum allocated memory during execution.…”

Section: Methodsmentioning

confidence: 99%

Engineering Practical Lempel-Ziv Tries

Arroyuelo

Cánovas

Fischer

et al. 2021

ACM J. Exp. Algorithmics

Self Cite

View full text Add to dashboard Cite

The Lempel-Ziv 78 ( LZ78 ) and Lempel-Ziv-Welch ( LZW ) text factorizations are popular, not only for bare compression but also for building compressed data structures on top of them. Their regular factor structure makes them computable within space bounded by the compressed output size. In this article, we carry out the first thorough study of low-memory LZ78 and LZW text factorization algorithms, introducing more efficient alternatives to the classical methods, as well as new techniques that can run within less memory space than the necessary to hold the compressed file. Our results build on hash-based representations of tries that may have independent interest.

show abstract

“…The pressure efficiency of these methods is limited up to their information entropy. Typical lossless data compression and reconstruction methods include Huffman coding [17], run-length coding [18], arithmetic coding [19], dictionary coding [20,21], and so on. The principles of them are referred to relevant references [17][18][19][20][21], which are not discussed here.…”

Section: Lossless Data Compression and Reconstruction Algorithmsmentioning

confidence: 99%

Two-level fusion big data compression and reconstruction framework combining second-generation wavelet and lossless compression

Zhang

2020

Complex Intell. Syst.

View full text Add to dashboard Cite

In view of the characteristics of big data, fuzziness, and real time of data acquisition and transmission in the fuzzy information system faced by aircraft health management, to reduce the load of airborne data processing and transmission system under the condition of limited airborne computing resources and strong time constraints, the data collected by the airborne system are first compressed, and the amount of data are reduced before transmission and reconstructed after transmission. In view of the situation that the compression ratio of primary data compression is too small and the compression time is too long for large-scale fuzzy systems to meet the transmission requirements of the system, this paper combines the advantages of lossy compression method which consumes less time and lossless compression method which has higher compression ratio, and innovatively proposes a two-level data compression and reconstruction framework combining lossy compression and lossless compression. The optimization analysis is carried out. Taking a real aero-engine health sample as an example, the validity, scientificity, and robustness of the proposed framework are verified by comparing with data compression and reconstruction algorithm based on redundant sparse representation and compressed sensing.

show abstract

LZ78 Compression in Low Main Memory Space

Cited by 6 publications

References 20 publications

Dynamic Path-Decomposed Tries

Dynamic Path-Decomposed Tries

Engineering Practical Lempel-Ziv Tries

Two-level fusion big data compression and reconstruction framework combining second-generation wavelet and lossless compression

Contact Info

Product

Resources

About