Abstract-We propose an information-theoretic framework for matrix completion. The theory goes beyond the low-rank structure and applies to general matrices of "low description complexity". Specifically, we consider random matrices X ∈ R m×n of arbitrary distribution (continuous, discrete, discretecontinuous mixture, or even singular). With S ⊆ R m×n an ε-support set of X, i.e., P[X ∈ S] ≥ 1 − ε, and dim B (S) denoting the lower Minkowski dimension of S, we show that k > dim B (S) measurements of the form Ai, X , with Ai denoting the measurement matrices, suffice to recover X with probability of error at most ε. The result holds for Lebesgue a.a. Ai and does not need incoherence between the Ai and the unknown matrix X. We furthermore show that k > dim B (S) measurements also suffice to recover the unknown matrix X from measurements taken with rank-one Ai, again this applies to a.a. rank-one Ai. Rank-one measurement matrices are attractive as they require less storage space than general measurement matrices and can be applied faster. Particularizing our results to the recovery of low-rank matrices, we find that k > (m + n − r)r measurements are sufficient to recover matrices of rank at most r. Finally, we construct a class of rank-r matrices that can be recovered with arbitrarily small probability of error from k < (m + n − r)r measurements.

We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals from a noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression. The main results of the present paper are a general achievability bound for the compression rate in the analog signal separation problem, an exact expression for the optimal compression rate in the case of signals that have mixed discrete-continuous distributions, and a new technique for showing that the intersection of generic subspaces with subsets of sufficiently small Minkowski dimension is empty. This technique can also be applied to obtain a simplified proof of a key result in Wu and Verdú (2010).

While spike trains are obviously not band-limited, the theory of super-resolution tells us that perfect recovery of unknown spike locations and weights from low-pass Fourier transform measurements is possible provided that the minimum spacing, ∆, between spikes is not too small. Specifically, for a measurement cutoff frequency of f c , Donoho [2] showed that exact recovery is possible if the spikes (on R) lie on a lattice and ∆ > 1/f c , but does not specify a corresponding recovery method. Candès and Fernandez-Granda [3, 4] provide a convex programming method for the recovery of periodic spike trains (i.e., spike trains on the torus T), which succeeds provably if ∆ > 2/f c and f c 128 or if ∆ > 1.26/f c and f c 10 3 , and does not need the spikes within the fundamental period to lie on a lattice. In this paper, we develop a theory of super-resolution from short-time Fourier transform (STFT) measurements. Specifically, we present a recovery method similar in spirit to the one in [3] for pure Fourier measurements. For a STFT Gaussian window function of width σ = 1/(4f c ) this method succeeds provably if ∆ > 1/f c , without restrictions on f c . Our theory is based on a measure-theoretic formulation of the recovery problem, which leads to considerable generality in the sense of the results being grid-free and applying to spike trains on both R and T. The case of spike trains on R comes with significant technical challenges. For recovery of spike trains on T we prove that the correct solution can be approximated-in weak-* topology-by solving a sequence of finite-dimensional convex programming problems.

We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals, modeled as general random vectors, from the noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on analog compression and encompasses, inter alia, inpainting, declipping, super-resolution, the recovery of signals corrupted by impulse noise, and the separation of (e.g., audio or video) signals into two distinct components. The main results we report are general achievability bounds for the compression rate, i.e., the number of measurements relative to the dimension of the ambient space the signals live in, under either measurability or Hölder continuity imposed on the separator. Furthermore, we find a matching converse for sources of mixed discrete-continuous distribution. For measurable separators our proofs are based on a new probabilistic uncertainty relation which shows that the intersection of generic subspaces with general sets of sufficiently small Minkowski dimension is empty. Hölder continuous separators are dealt with by introducing the concept of regularized probabilistic uncertainty relations. The probabilistic uncertainty relations we develop are inspired by embedding results in dynamical systems theory due to Sauer et al. (1991) and-conceptually-parallel classical Donoho-Stark and Elad-Bruckstein uncertainty principles at the heart of compressed sensing theory. Operationally, the new uncertainty relations take the theory of sparse signal separation beyond traditional sparsity-as measured in terms of the number of non-zero entries-to the more general notion of low description complexity as quantified by Minkowski dimension. Finally, our approach also allows to significantly strengthen key results in Wu and Verdú (2010).

No abstract