On the bit-complexity of Lempel-Ziv compression

Abstract

One of the most famous and investigated lossless datacompression schemes is the one introduced by Lempel and Ziv about 30 years ago [37]. This compression scheme is known as "dictionary-based compressor" and consists of squeezing an input string by replacing some of its substrings with (shorter) codewords which are actually pointers to a dictionary of phrases built as the string is processed. Surprisingly enough, although many fundamental results are nowadays known about the speed and effectiveness of this compression process (see e.g. [23,28] and references therein), "we are not aware of any parsing scheme that achieves optimality when the LZ77-dictionary is in use under any constraint on the codewords other than being of equal length" [28, pag. 159]. Here optimality means to achieve the minimum number of bits in compressing each individual input string, without any assumption on its generating source. In this paper we investigate three issues pertaining to the bit-complexity of LZ-based compressors, and we design algorithms which achieve bit-optimality in the compressed output size by taking efficient/optimal time and optimal space. These theoretical results will be sustained by some experiments that will compare our novel LZ-based compressors against the most popular compression tools (like gzip, bzip2) and state-of-theart compressors (like the booster of [14,13]).

Range Predecessor and Lempel-Ziv Parsing

“…For this problem we describe an algorithm using O(n log σ) bits of space and O(n(log log σ + log σ/ √ log n)) (O(n(1 + log σ/ √ log n)) if we allow randomization) time. This significantly improves on the algorithm of Ferragina et al [15], which runs in O(n(1 + log σ/ log log n)) and anyway requires O(n log n) bits of space. 3.…”

“…Recently, Larsson [31] showed how to compute the rightmost parsing online in the same O(n log n) time and space bounds as Amir et al. Currently the best prior result for rightmost parsing is an O(n log n) space and O(n + n log σ/ log log n) time algorithm due to Ferragina, Nitto and Venturini [15]. We provide a faster algorithm that uses significantly less space.…”

“…We will now apply the construction for range predecessor detailed above to obtain a faster algorithm for computing rightmost previous occurrences of LZ phrases. An algorithm by Ferragina et al [15] achieves O(n(1 + log σ/ log log n)) time, but requires O(n log n) bits of space that can not trivially be reduced.…”

“…If p i < i is the largest possible for every phrase then we call the parsing rightmost. In their study on the bit-complexity of LZ compression, Ferragina et al [15] showed that the rightmost parsing can lead to encodings asymptotically smaller than what is achievable with other choices of p i .…”

Compressed Data Structures for Strings

“…LZ77 is a compression scheme that has been introduced by Lempel and Ziv about 30 years ago [Ziv and Lempel (1977)]. The content of this chapter is based on papers [Ferragina et al (2009d[Ferragina et al ( , 2013a]. In Chap.…”