Entropy and Source Coding for Integer-Dimensional Singular Random Variables

Koliander, Günther; Pichler, Georg; Riegler, Erwin; Hlawatsch, Franz

doi:10.1109/tit.2016.2604248

Cited by 16 publications

(16 citation statements)

References 27 publications

(99 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A slightly weaker version of this statement, valid for Lipschitz mappings, was derived previously in [23,Lemma 4].…”

Section: Rectifiable Sets and Rectifiable Random Vectorsmentioning

confidence: 92%

“…We particularize our achievability result to s-rectifiable random vectors x as introduced in [23]; these are random vectors supported on countable unions of s-dimensional C 1submanifolds of R m and of absolutely continuous-with respect to s-dimensional Hausdorff measure-distribution. Countable unions of C 1 -submanifolds include numerous signal models prevalent in the compressed sensing literature, namely, the standard union of subspaces model underlying much of compressed sensing theory [24], [25] and spectrumblind sampling [26], [27], smooth submanifolds [28], blocksparsity [29]- [31], and low-rank matrices as considered in the matrix completion problem [32]- [34].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Lossless Analog Compression

Alberti

Bölcskei

Lellis

et al. 2019

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary random vectors x ∈ R m from the noiseless linear measurements y = Ax with measurement matrix A ∈ R n×m . Our theory is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero error probability. Specifically, our achievability result states that, for Lebesgue-almost all A, the random vector x can be recovered with zero error probability provided that n > K(x), where K(x) is given by the infimum of the lower modified Minkowski dimension over all support sets U of x (i.e., sets U ⊆ R m with P[x ∈ U] = 1). We then particularize this achievability result to the class of s-rectifiable random vectors as introduced in Koliander et al. (2016); these are random vectors of absolutely continuous distribution-with respect to the sdimensional Hausdorff measure-supported on countable unions of s-dimensional C 1 -submanifolds of R m . Countable unions of C 1 -submanifolds include essentially all signal models used in the compressed sensing literature such as the standard union of subspaces model underlying much of compressed sensing theory and spectrum-blind sampling, smooth submanifolds, block-sparsity, and low-rank matrices as considered in the matrix completion problem. Specifically, we prove that, for Lebesgue-almost all A, s-rectifiable random vectors x can be recovered with zero error probability from n > s linear measurements. This threshold is, however, found not to be tight as exemplified by the construction of an s-rectifiable random vector that can be recovered with zero error probability from n < s linear measurements. Motivated by this observation, we introduce the new class of s-analytic random vectors, which admit a strong converse in the sense of n ≥ s being necessary for recovery with probability of error smaller than one. The central conceptual tools in the development of our theory are geometric measure theory and the theory of real analytic functions.

show abstract

“…A slightly weaker version of this statement, valid for Lipschitz mappings, was derived previously in [23,Lemma 4].…”

Section: Rectifiable Sets and Rectifiable Random Vectorsmentioning

confidence: 92%

mentioning

confidence: 99%

Lossless Analog Compression

Alberti

Bölcskei

Lellis

et al. 2019

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…This was introduced by Csiszár in [5], see also Eq. ( 8) in [7]. Remark that the set where f = 0, hence log(f ) = −∞, is ρ-negligible.…”

Section: Definition and Aepmentioning

confidence: 99%

Entropy under disintegrations

Vigneaux¹

2021

Preprint

View full text Add to dashboard Cite

We consider the differential entropy of probability measures absolutely continuous with respect to a given σ-finite reference measure on an arbitrary measurable space. We state the asymptotic equipartition property in this general case; the result is part of the folklore but our presentation is to some extent novel. Then we study a general framework under which such entropies satisfy a chain rule: disintegrations of measures. We give an asymptotic interpretation for conditional entropies in this case. Finally, we apply our result to Haar measures in canonical relation.

show abstract

“…where (40) is by [49,Equation (3.6)] and in (42) we used (33). We can hence conclude that γ p,d is ρ-subregular of dimension d − p. The corresponding subregularity constants are…”

mentioning

confidence: 96%

“…For continuous X of finite differential entropy under the difference distortion function ρ(x, y) = x − y k , where • is a semi-norm and k ∈ (0, ∞), the Shannon lower bound is known explicitly [63, Section VI] and, provided that X additionally satisfies a certain moment constraint, is tight as D → 0 [43,39]. For the class of m-rectifiable random variables [40,Definition 11], a Shannon lower bound was reported recently in [40,Theorem 55]. This bound is, however, not in explicit form and depends on a parametrized integral.…”

mentioning

confidence: 99%

Lossy Compression of General Random Variables

Riegler¹,

Bölcskei²,

Koliander³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in the modeling of Ethernet traffic. Our main contributions are bounds on the rate-distortion function and the quantization error. These bounds are very general and essentially only require the existence of reference measures satisfying certain regularity conditions in terms of small ball probabilities. To illustrate the wide applicability of our results, we particularize them to random variables taking values in i) manifolds, namely, hyperspheres and Grassmannians, and ii) self-similar sets characterized by iterated function systems satisfying the weak separation property.

show abstract

Entropy and Source Coding for Integer-Dimensional Singular Random Variables

Cited by 16 publications

References 27 publications

Lossless Analog Compression

Lossless Analog Compression

Entropy under disintegrations

Lossy Compression of General Random Variables

Contact Info

Product

Resources

About