On Universal Coding of Unordered Data

Varshney, Lav R.; Goyal, Vivek K

doi:10.1109/ita.2007.4357578

Cited by 10 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 This paper now explores similarly fundamental compression models for input objects that aren't sequences. Compression of non-sequential data was considered by Varshney and Goyal [6,7,8], and this paper follows up on their work. The models in this paper have been chosen primarily for simplicity and clarity of presentation; more elaborate models for combinatorial objects can be found e.g.…”

Section: Sequencesmentioning

confidence: 99%

Compressing Combinatorial Objects

Steinruecken

2016

2016 Data Compression Conference (DCC)

View full text Add to dashboard Cite

Most of the world's digital data is currently encoded in a sequential form, and compression methods for sequences have been studied extensively. However, there are many types of nonsequential data for which good compression techniques are still largely unexplored. This paper contributes insights and concrete techniques for compressing various kinds of nonsequential data via arithmetic coding, and derives re-usable probabilistic data models from fairly generic structural assumptions. Near-optimal compression methods are described for certain types of permutations, combinations and multisets; and the conditions for optimality are made explicit for each method.

show abstract

Section: Sequencesmentioning

confidence: 99%

Compressing Combinatorial Objects

Steinruecken

2016

2016 Data Compression Conference (DCC)

View full text Add to dashboard Cite

show abstract

“…Define the spacing in interval as (37) where is a parameter to be optimized. Similarly to (19), the interval cardinality here is (38) In a similar manner to the definition of in (20), we define (39) The cardinality of is (40) We now perform the encoding similarly to the small case, where we allow quantization to nonzero values to the components of up to . (This is more than needed but is possible since .)…”

Section: Upper Bounds For Small and Large Alphabetsmentioning

confidence: 99%

“…Further study of universal compression of patterns [13], [22], [23], [31], [35] (and subsequently to the work in this paper in [2]) provided various lower and upper bounds to various forms of redundancy in universal compression of patterns. Another related study is that of compression of data, where the order of the occurring data symbols is not important, but their types and empirical counts are [39], [40].…”

Section: Introductionmentioning

confidence: 99%

Universal Source Coding for Monotonic and Fast Decaying Monotonic Distributions

Shamir

2013

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We study universal compression of sequences generated by monotonic distributions. We show that for a monotonic distribution over an alphabet of size , each probability parameter costs essentially bits, where is the coded sequence length, as long as . Otherwise, for , the total average sequence redundancy is bits overall. We then show that there exists a sub-class of monotonic distributions over infinite alphabets for which redundancy of bits overall is still achievable. This class contains fast decaying distributions, including many distributions over the integers such as the family of Zipf distributions and geometric distributions. For some slower decays, including other distributions over the integers, redundancy of bits overall is achievable. A method to compute specific redundancy rates for such distributions is derived. The results are specifically true for finite entropy monotonic distributions. Finally, we study individual sequence redundancy behavior assuming a sequence is governed by a monotonic distribution. We show that for sequences whose empirical distributions are monotonic, individual redundancy bounds even tighter than those in the average case can be obtained. The relation of universal compression with monotonic distributions to universal compression of patterns of sequences is demonstrated.

show abstract

“…they have prefix-free equivalents. Later work by Varshney and Goyal [5][6][7] lower bounded the entropy of a multiset by decomposing it into the entropy of a deterministic ordering of the elements and the entropy of a uniform distribution over all possible permutations [5, Eq. (2)].…”

Section: Introductionmentioning

confidence: 99%

Predictive Coding for Lossless Dataset Compression

Barowsky

Alexander

Calmon

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Lossless compression of datasets is a problem of significant theoretical and practical interest. It appears naturally in the task of storing, sending, or archiving large collections of information for scientific research. We can greatly improve encoding bitrate if we allow the compression of the original dataset to decompress to a permutation of the data. We prove the equivalence of dataset compression to compressing a permutation-invariant structure of the data and implement such a scheme via predictive coding. We benchmark our compression procedure against state-of-the-art compression utilities on the popular machine-learning datasets MNIST and CIFAR-10 and outperform for multiple parameter sets.

show abstract

On Universal Coding of Unordered Data

Cited by 10 publications

References 18 publications

Compressing Combinatorial Objects

Compressing Combinatorial Objects

Universal Source Coding for Monotonic and Fast Decaying Monotonic Distributions

Predictive Coding for Lossless Dataset Compression

Contact Info

Product

Resources

About