Divergence and the construction of variable-to-variable-length lossless codes by source-word extensions

Abstract

Variable-to-variable-length lossless codes a r e described using dual leaf-linked trees: one specifying t h e parsing of t h e source symbols into source words and t h e other specifying t h e formation of code words from code symbols. Compression exceeds entropy by t h e amount of t h e informational divergence, between source words and code words, divided by t h e expected source-word length. The asymptotic optimality of Tunstall or Huffman codes derives from t h e bounding of divergence while t h e expected source-word length is made arbitrarily large. A heuristic extension scheme is asymptotically optimal but also acts to reduce t h e divergence by retaining those source words which a r e well matched t o their corresponding code words. DUAL-TREE ENTROPY CODING [3][4]Variable-length source words are specified by a complete, proper, labelled, a-ary parse tree. Its symbol alphabet SW, with lSwl = a, is that of the source sequence w. Its index alphabet is Iw = (0, ,7' -1 ) . To parse one source word at the encoder, symbols of the source sequence specify a path from the root node of the parse tree until a leaf node is reached. To reconstruct one source word at the decoder, a leaf node is specified and the source symbols are recreated by following its unique path from the parse tree root.Variable-length code words are similarly specified by a complete, proper, labelled, p-ary code tree. Its symbol alphabet Sz, with lSzl = p, is that of the channel sequence z. Its index alphabet is I z = (0, --,7). Code words are parsed from the channel sequence by the decoder or constructed by the encoder using root-to-leaf paths in the code tree.A one-to-one interface is established between the 7' words of the parse tree and a subset of the 7 words of the code tree. To avoid unused code words, it is assumed that the parse tree has 7 -7' fake leaf nodes representing arbitrary source words of tThis work

Properties and Design of Variable-to-Variable Length Codes

“…Given such constraints, the question arises, how minimum redundancy codes can be found? In the related literature, several algorithms for V2V code design evolved [6][7][8][21][22][23]. However, algorithms for finding minimum redundancy codes are all based on exhaustive search.…”

“…Note that codes with minimum redundancy are also codes with minimum average code length and vice versa according to Equation(8).…”

On the Construction of (Explicit) Khodak's Code and Its Analysis

“…There is scarcely any literature on VV codes with a few exceptions such as [9,10,14,18]. The most interesting, as already mentioned, is a thirty year old work by Khodak [14].…”

“…To the best of our knowledge not much was done since then, except that Fabris [9] (cf. also [10,18]) analyzed Tunstall-Huffman VV code and provided a simple bound on their redundancy rate.…”

Length-limited variable-to-variable length codes for high-performance entropy coding

“…The most common way of creating a VV code uses a variant of the Tunstall extension approach to create the parse tree, and uses the Huffman algorithm to generate the code words. To our knowledge most work in VV code algorithms (see [3,4,10,11] for example) focuses on selecting a leaf for extension such that the resulting parse tree gives the best coding rate. The process of determining which leaf to extend is notoriously difficult, and currently it seems that an exhaustive search is the only possible method for finding an optimal code.…”