Rate-Distortion Performance of Lossy Compressed Sensing of Sparse Sources

Abstract

We investigate lossy compressed sensing (CS) of a hidden, or remote, source, where a sensor observes a sparse information source indirectly. The compressed noisy measurements are communicated to the decoder for signal reconstruction with the aim to minimize the mean square error distortion. An analytically tractable lower bound to the remote rate-distortion function (RDF), i.e., the conditional remote RDF, is derived by providing support side information to the encoder and decoder. For this setup, the best encoder separates into an estimation step and a transmission step. A variant of the Blahut-Arimoto algorithm is developed to numerically approximate the remote RDF. Furthermore, a novel entropy coding based quantized CS method is proposed. Numerical results illustrate the main rate-distortion characteristics of the lossy CS, and compare the performance of practical quantized CS methods against the proposed limits.Index Terms-Remote source coding, conditional ratedistortion theory, side information, Blahut-Arimoto algorithm, wireless sensor networks.

Practical Compression Methods for Quantized Compressed Sensing

“…based QCS algorithms have been devised in, e.g., [14,21,[27][28][29][30]. The related informationtheoretic studies include [14,31,32].…”

“…For the QCS setup in Fig 1, the compression limit, i.e., the minimum achievable rate for a given distortion , is given by the remote rate-distortion function (RDF) of source , denoted as . While the closed-form solution of remains open, the RD performance of QCS can be assessed by the techniques derived in [14]. Thus, we establish information-theoretic benchmarks for the proposed QCS algorithms of Section IV by evaluating:…”

“…1) The analytical lower bound to , termed the conditional remote RDF , given in [14,Theorem 1]. is based on having support side information at the encoder and decoder.…”

“…In particular, the strategy 3) contains several novel methods which are empirically shown to achieve competent RD performance with low complexity, beneficial to energy-limited IoT sensor applications. Simulation results demonstrate the RD performances of different practical QCS schemes and compare them to the compression limit -the remote rate-distortion functionwhich is evaluated through an analytical lower bound and a modified Blahut-Arimoto algorithm proposed in [14].…”

Gaussian Approximation of Quantization Error for Estimation from Compressed Data

“…By applying our main results to estimation with the approximate message passing (AMP) algorithm [24], we provide an exact asymptotic characterisation of the MSE in recovering θ from a lossy compressed version of X obtained using bitrate-R random spherical coding. Versions of this compression and estimation problem for other type of lossy compression codes and estimators were considered in [25]- [28].…”