Efficient and Compact Representations of Prefix Codes

Abstract

Most of the attention in statistical compression is given to the space used by the compressed sequence, a problem completely solved with optimal prefix codes. However, in many applications, the storage space used to represent the prefix code itself can be an issue. In this paper we introduce and compare several techniques to store prefix codes. Let N be the sequence length and n be the alphabet size. Then a naive storage of an optimal prefix code uses O(n log n) bits. Our first technique shows how to use O(n log log(N/n)) bits to store the optimal prefix code. Then we introduce an approximate technique that, for any 0 < < 1/2, takes O(n log log(1/ )) bits to store a prefix code with average codeword length within an additive of the minimum. Finally, a second approximation takes, for any constant c > 1, O n 1/c log n bits to store a prefix code with average codeword length at most c times the minimum. In all cases, our data structures allow encoding and decoding of any symbol in O(1) time. We experimentally compare our new techniques with the state of the art, showing that we achieve 6-8-fold space reductions, at the price of a slower encoding (2.5-8 times slower) and decoding (12-24 times slower). The approximations further reduce this space and improve the time significantly, up to recovering the speed of classical implementations, for a moderate penalty in the average code length. As a byproduct, we compare various heuristic, approximate, and optimal algorithms to generate length-restricted codes, showing that the optimal ones are clearly superior and practical enough to be implemented.

A Cipher Based on Prefix Codes

“…Some technicalities about possible coders and decoders for P -codes can be further found in [ 18 ]. Information on effective decoding algorithms can be found in [ 19 , 20 ] and on memory-efficient representation of prefix codes can be found in [ 21 ].…”

Efficient and Compact Representations of Some Non-canonical Prefix-Free Codes

“…We have run experiments to compare the solution of Theorem 2 (referred to as WMM in the sequel, for Wavelet Matrix Model) with the only previous encoding, that is, the one used by Claude et al [1] (denoted by TABLE). Note that our codes are not canonical, so other solutions [5] do not apply. Claude et al [1] use for encoding a single table of σL bits storing the code of each symbol, and thus they easily encode in constant time.…”

“…If the alphabet consists of σ characters and the maximum codeword length is L, then we can build an O(σ log L)-bit data structure with O(log L) query time that, given a character, returns its codeword's length and rank among codewords of that length, or vice versa. If L is at most a constant times the size of a machine word (which it is when we are considering, e.g., Huffman codes for strings in the RAM model) then in theory we can make the predecessor search and the data structure's queries constant-time, meaning we can encode and decode in constant time [5].…”

“…Interestingly, since the mapping between symbols and leaves is fixed, alphabetic codes need only to store the tree topology, which can be represented more succinctly than optimal prefix-free codes, in 2σ + o(σ) bits [9], so that encoding and decoding can still be done in time O(ℓ). There are no, however, equivalents to the faster encoding/decoding methods used on canonical codes [5].…”

Space-Efficient Huffman Codes Revisited

“…The same idea is presented perhaps more lucidly in [25, Section 2.6.3], with an explicit claim that this representation requires σ lg σ + O(lg 2 n) bits 2 and achieves O(lg lg n) time per codeword. Given ℓ max is the maximum length of the codewords, an improvement, both in space and time, has been achieved by Gagie et al [13], who gave a representation of the canonical Huffman tree within…”