Lossless Analog Compression

Abstract

We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary random vectors x ∈ R m from the noiseless linear measurements y = Ax with measurement matrix A ∈ R n×m . Our theory is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero error probability. Specifically, our achievability result states that, for Lebesgue-almost all A, the random vector x can be recovered with zero error probability provided that n > K(x), where K(x) is given by the infimum of the lower modified Minkowski dimension over all support sets U of x (i.e., sets U ⊆ R m with P[x ∈ U] = 1). We then particularize this achievability result to the class of s-rectifiable random vectors as introduced in Koliander et al. (2016); these are random vectors of absolutely continuous distribution-with respect to the sdimensional Hausdorff measure-supported on countable unions of s-dimensional C 1 -submanifolds of R m . Countable unions of C 1 -submanifolds include essentially all signal models used in the compressed sensing literature such as the standard union of subspaces model underlying much of compressed sensing theory and spectrum-blind sampling, smooth submanifolds, block-sparsity, and low-rank matrices as considered in the matrix completion problem. Specifically, we prove that, for Lebesgue-almost all A, s-rectifiable random vectors x can be recovered with zero error probability from n > s linear measurements. This threshold is, however, found not to be tight as exemplified by the construction of an s-rectifiable random vector that can be recovered with zero error probability from n < s linear measurements. Motivated by this observation, we introduce the new class of s-analytic random vectors, which admit a strong converse in the sense of n ≥ s being necessary for recovery with probability of error smaller than one. The central conceptual tools in the development of our theory are geometric measure theory and the theory of real analytic functions.

Metric mean dimension and analog compression

“…However, it does not provide Hölder inverse and argues that it cannot exist in general in the probabilistic context. Similar results for the modied lower box-counting dimension have been obtained also in [44].…”

“…Applying Jensen's inequality (see e.g. [31,Theorem 3.3]) yields (44). As k 0 ≤ n 0 , inequality (45) follows.…”

Almost Lossless Analog Signal Separation and Probabilistic Uncertainty Relations

“…3.4]), we can assume, w.l.o.g., that U is compact. 5 For the detailed arguments leading to this statement, we refer to [22].…”

Lossy Compression of General Random Variables

“…This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion (R-D) theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing [4,9,8,1,54,11], machine learning [21,45], image processing [44,57], directional statistics [47], and handwritten digit recognition [31]. Fractal sets find application in image compression and in modeling of Ethernet traffic [41].…”

“…In R-D theory [56,6,28,29,15], one is interested in the characterization of the ultimate limits on the compression of sequences of random variables under a distortion constraint, here expressed in terms of expected average distortion. Specifically, let (A, A ) and (B, B) be measurable spaces equipped with a measurable function σ : A × B → [0, ∞], henceforth referred to as distortion function, 1 and let (X i ) i∈N be a sequence of random variables taking values in (A, A ). For every ℓ ∈ N, measurable mappings g (ℓ) : A ℓ → B ℓ with |g (ℓ) (A ℓ )| < ∞ are referred to as source codes of length ℓ.…”