Average case universal lossless compression with unknown alphabets

“…Several recent works (see, e.g., [1], [3]- [4], [6], [8], [9]) have considered universal compression for patterns of independently identically distributed (i.i.d.) sequences.…”

Section: Introductionmentioning

confidence: 99%

“…(This is, of course, related to the fact that we loose some information by coding the pattern instead of the actual sequence.) The universal average case was then studied in [6], [8], [9], where redundancy bounds for average case universal compression of patterns were derived.…”

Section: Introductionmentioning

confidence: 99%

Bounds on the entropy of patterns of I.I.D. sequences

Shamir

2005

IEEE Information Theory Workshop, 2005.

Self Cite

View full text Add to dashboard Cite

Abstract-Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first occurrence. If the alphabet of a source that generated a sequence is unknown, the inevitable cost of coding the unknown alphabet symbols can be exploited to create the pattern of the sequence. This pattern can in turn be compressed by itself. The bounds derived here are functions of the i.i.d. source entropy, alphabet size, and letter probabilities. It is shown that for large alphabets, the pattern entropy must decrease from the i.i.d. one. The decrease is in many cases more significant than the universal coding redundancy bounds derived in prior works. The pattern entropy is confined between two bounds that depend on the arrangement of the letter probabilities in the probability space. For very large alphabets whose size may be greater than the coded pattern length, all low probability letters are packed into one symbol. The pattern entropy is upper and lower bounded in terms of the i.i.d. entropy of the new packed alphabet. Correction terms, which are usually negligible, are provided for both upper and lower bounds.

show abstract

“…It was first introduced byÅberg in [8] as a solution to the multi-alphabet coding problem, where the message x contains only a small subset of the known alphabet A. It was further studied and motivated in a series of articles by Shamir [9][10][11][12] and by Jevtić, Orlitsky, Santhanam and Zhang [13][14][15][16] for practical applications: the alphabet is unknown and has to be transmitted separately anyway (for instance, transmission of a text in an unknown language), or the alphabet is very large in comparison to the message (consider the case of images with k = 2 24 colors, or texts when taking words as the alphabet units).…”

Section: Dictionary and Patternmentioning

confidence: 99%

A Lower-Bound for the Maximin Redundancy in Pattern Coding

Garivier

2009

Entropy

View full text Add to dashboard Cite

We show that the maximin average redundancy in pattern coding is eventually larger than 1.84for messages of length n. This improves recent results on pattern redundancy, although it does not fill the gap between known lower-and upper-bounds. The pattern of a string is obtained by replacing each symbol by the index of its first occurrence. The problem of pattern coding is of interest because strongly universal codes have been proved to exist for patterns while universal message coding is impossible for memoryless sources on an infinite alphabet. The proof uses fine combinatorial results on partitions with small summands.

show abstract

Average case universal lossless compression with unknown alphabets

Cited by 16 publications

References 26 publications

Universal Lossless Compression With Unknown Alphabets—The Average Case

Universal Lossless Compression With Unknown Alphabets—The Average Case

Bounds on the entropy of patterns of I.I.D. sequences

A Lower-Bound for the Maximin Redundancy in Pattern Coding

Contact Info

Product

Resources

About