On the Construction of (Explicit) Khodak's Code and Its Analysis

Abstract

Variable-to-variable codes are very attractive yet not well understood data compression schemes. In 1972 Khodak claimed to provide upper and lower bounds for the achievable redundancy rate, however, he did not offer explicit construction of such codes. In this paper, we first present a constructive and transparent proof of Khodak's result showing that for memoryless sources there exists a code with the average redundancy bounded by D −5/3 , where D is the average delay (e.g., the average length of a dictionary entry). We also describe an algorithm that constructs a variable-to-variable length code with a small redundancy rate for large D. Then, we discuss several generalizations. We prove that the worst case redundancy does not exceed D −4/3 . Furthermore we provide similar upper bounds for Markov sources (of order 1). Finally, we consider bounds that are valid for almost all memoryless and Markov sources for which the set of exceptional source parameters has zero measure. In particular, for all memoryless sources outside this exceptional class, we prove there exists a variable-to-variable code with the average redundancy rate bounded by D −4/3−m/3+ε and the worst case redundancy rate bounded by D −1−m/3+ε , where m is the cardinality of the alphabet. We complete our analysis with a lower bound showing that for all variable-to-variable codes the average and the worst case redundancy rates are at least D −2m−1−ε for almost all memoryless sources in the sense that the set of exceptional source parameters has zero measure. We prove these results using techniques of Diophantine approximations.Index Terms: Variable-to-variable length codes, average and maximal redundancy rates, metric Diophantine approximations. * A preliminary version of some parts of this paper was presented at

Information Transfer Ion Wireless Networks

“…For example, in [19] an error resilient Lempel-Ziv'77 scheme has been designed that also can be used for information hiding. In [23] and [33] we propose a precise analysis of variable-to-variable code that achieves very small redundancy. This was thoroughly discussed in our invited paper [28].…”

Efficient implementation of the generalized Tunstall code generation algorithm

“…This not only aids in randomly accessing portions of the compression sequence, but also in synchronization: Because the size of output blocks is fixed, simple symbol errors do not propagate beyond the set of input symbols a given output block represents. Huffman codes and variable-to-variable-length codes (e.g., those in [6]) do not share this property.…”

Redundancy of Lossless Data Compression for Known Sources by Analytic Methods

“…He did not offer, however, an explicit VV code construction. We will remedy this situation and follow [16].…”