Covering a Ball with Smaller Equal Balls in ?n

Abstract-This paper studies the minimum achievable source coding rate as a function of blocklength n and probability ǫ that the distortion exceeds a given level d. Tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength. For stationary memoryless sources with separable distortion, the minimum rate achievable is shown to be closelyis the rate dispersion, a characteristic of the source which measures its stochastic variability, and Q −1 (·) is the inverse of the standard Gaussian complementary cdf.

show abstract

“…two are due to the recent improvement by Verger-Gaugry [38]. An immediate corollary to Theorem 38 is the following:…”

Section: Proofmentioning

confidence: 95%

Fixed-length lossy compression in the finite blocklength regime: Discrete memoryless sources

Kostina

Verdú

2011

2011 IEEE International Symposium on Information Theory Proceedings

View full text Add to dashboard Cite

show abstract

“…Such a probabilistic proof was independently proposed by M. Koucký. The authors would also like to thank the referees for their constructive comments; one referee pointed out that yet another example would be the case of Euclidean balls with the usual Euclidean distance, where the important Property 4 is proved in for example [23].…”

Section: Acknowledgmentmentioning

confidence: 99%

Rate Distortion and Denoising of Individual Data Using Kolmogorov Complexity

Vereshchagin

Vitányi

2010

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-We examine the structure of families of distortion balls from the perspective of Kolmogorov complexity. Special attention is paid to the canonical rate-distortion function of a source word which returns the minimal Kolmogorov complexity of all distortion balls containing that word subject to a bound on their cardinality. This canonical rate-distortion function is related to the more standard algorithmic rate-distortion function for the given distortion measure. Examples are given of list distortion, Hamming distortion, and Euclidean distortion. The algorithmic rate-distortion function can behave differently from Shannon's rate-distortion function. To this end, we show that the canonical rate-distortion function can and does assume a wide class of shapes (unlike Shannon's); we relate low algorithmic mutual information to low Kolmogorov complexity (and consequently suggest that certain aspects of the mutual information formulation of Shannon's rate-distortion function behave differently than would an analogous formulation using algorithmic mutual information); we explore the notion that low Kolmogorov complexity distortion balls containing a given word capture the interesting properties of that word (which is hard to formalize in Shannon's theory) and this suggests an approach to denoising.

show abstract

“…However, because the Gaussian source is continuous, we need to modify the type covering lemma mentioned above, as it only applies to discrete sources there. We apply the sphere covering theorem [15] multiple times to establish a Gaussian type covering lemma for the successive refinement problem. To subsequently apply this lemma to calculate the joint excess-distortion probability, we need to define the notion of Gaussian types (cf.…”

Section: A Main Contributionsmentioning

confidence: 99%

Second-Order and Moderate Deviation Asymptotics for Successive Refinement

Zhou

Tan

Motani

2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We derive the optimal second-order coding region and moderate deviations constant for successive refinement source coding with a joint excess-distortion probability constraint. We consider two scenarios: (i) a discrete memoryless source (DMS) and arbitrary distortion measures at the decoders and (ii) a Gaussian memoryless source (GMS) and quadratic distortion measures at the decoders. For a DMS with arbitrary distortion measures, we prove an achievable second-order coding region, using type covering lemmas by Kanlis and Narayan and by No, Ingber and Weissman. We prove the converse using the perturbation approach by Gu and Effros. When the DMS is successively refinable, the expressions for the second-order coding region and the moderate deviations constant are simplified and easily computable. For this case, we also obtain new insights on the second-order behavior compared to the scenario where separate excess-distortion proabilities are considered. For example, we describe a DMS, for which the optimal second-order region transitions from being characterizable by a bivariate Gaussian to a univariate Gaussian, as the distortion levels are varied. We then consider a GMS with quadratic distortion measures. To prove the direct part, we make use of the sphere covering theorem by Verger-Gaugry, together with appropriately-defined Gaussian type classes. To prove the converse, we generalize Kostina and Verdú's one-shot converse bound for point-to-point lossy source coding. We remark that this proof is applicable to general successively refinable sources. In the proofs of the moderate deviations results for both scenarios, we follow a strategy similar to that for the second-order asymptotics and use the moderate deviations principle.

show abstract

Covering a Ball with Smaller Equal Balls in ?n

Cited by 55 publications

References 12 publications

Fixed-length lossy compression in the finite blocklength regime: Discrete memoryless sources

Fixed-length lossy compression in the finite blocklength regime: Discrete memoryless sources

Rate Distortion and Denoising of Individual Data Using Kolmogorov Complexity

Second-Order and Moderate Deviation Asymptotics for Successive Refinement

Contact Info

Product

Resources

About