Sparse regression codes for multi-terminal source and channel coding

Sarkar

IEEE Trans. Inform. Theory

2014

Self Cite

We propose computationally efficient encoders and decoders for lossy compression using a Sparse Regression Code. The codebook is defined by a design matrix and codewords are structured linear combinations of columns of this matrix. The proposed encoding algorithm sequentially chooses columns of the design matrix to successively approximate the source sequence. It is shown to achieve the optimal distortion-rate function for i.i.d Gaussian sources under the squared-error distortion criterion. For a given rate, the parameters of the design matrix can be varied to trade off distortion performance with encoding complexity. An example of such a trade-off as a function of the block length n is the following. With computational resource (space or time) per source sample of O((n/ log n) 2 ), for a fixed distortion-level above the Gaussian distortion-rate function, the probability of excess distortion decays exponentially in n. The Sparse Regression Code is robust in the following sense: for any ergodic source, the proposed encoder achieves the optimal distortion-rate function of an i.i.d Gaussian source with the same variance. Simulations show that the encoder has good empirical performance, especially at low and moderate rates. ).Communicated by M. Elad, Associate Editor for Signal Processing. 1 The error exponent of a compression code measures how fast the probability of excess distortion decays to zero with growing block length.The Sparse Regression codebook is constructed based on the statistical framework of high-dimensional linear regression, and was proposed recently by Barron and Joseph for communication over the AWGN channel at rates approaching the channel capacity [4], [5]. The codewords are sparse linear combinations of columns of an n × N design matrix or 'dictionary', where n is the block-length and N is a loworder polynomial in n. This structure enables the design of computationally efficient encoders based on sparse approximation ideas (e.g., [6], [7]). We propose one such encoding algorithm and analyze it performance.

Section: Discussionmentioning

confidence: 99%

Section: Remarksmentioning

confidence: 99%

See 1 more Smart Citation

Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding

Sarkar

IEEE Trans. Inform. Theory

2014

Self Cite

“…In [10], [11], we showed that SPARCs also attain the optimal rate-distortion function of Gaussian sources with feasible algorithms. Further, we showed in [12] that the source and channel coding modules can be combined to implement superposition and binning, which are key ingredients of several multi-terminal source and channel coding problems. Thus SPARCs offer a promising framework to build fast, rateoptimal codes for a variety of problems in network information theory.…”

Section: Introductionmentioning

confidence: 99%

Sparse Regression codes: Recent results and future directions

2013 IEEE Information Theory Workshop (ITW)

2013

Self Cite

Sparse Superposition or Sparse Regression codes were recently introduced by Barron and Joseph for communication over the AWGN channel. The code is defined in terms of a design matrix; codewords are linear combinations of subsets of columns of the matrix. These codes achieve the AWGN channel capacity with computationally feasible decoding. We have shown that they also achieve the optimal rate-distortion function for Gaussian sources. Further, the sparse regression codebook has a partitioned structure that facilitates random binning and superposition. In this paper, we review existing results concerning Sparse Regression codes and discuss directions for future research.

“…This monograph discusses a class of codes for such Gaussian models called Sparse Superposition Codes or Sparse Regression Codes (SPARCs). These codes were introduced by Barron and Joseph [16,65] for efficient communication over AWGN channels, but have since also been used for lossy compression [112,113] and multi-terminal communication [114]. Our goal in this monograph is to provide a unified and comprehensive view of SPARCs, covering theory, algorithms, as well as practical implementation aspects.…”

Section: Introductionmentioning

confidence: 99%

Sparse Regression Codes

FNT in Communications and Information Theory

Barron

2019

Self Cite

Developing computationally-efficient codes that approach the Shannon-theoretic limits for communication and compression has long been one of the major goals of information and coding theory. There have been significant advances towards this goal in the last couple of decades, with the emergence of turbo codes, sparse-graph codes, and polar codes. These codes are designed primarily for discrete-alphabet channels and sources. For Gaussian channels and sources, where the alphabet is inherently continuous, Sparse Superposition Codes or Sparse Regression Codes (SPARCs) are a promising class of codes for achieving the Shannon limits.This monograph provides a unified and comprehensive over-view of sparse regression codes, covering theory, algorithms, and practical implementation aspects. The first part of the monograph focuses on SPARCs for AWGN channel coding, and the second part on SPARCs for lossy compression (with squared error distortion criterion). In the third part, SPARCs are used to construct codes for Gaussian multi-terminal channel and source coding models such as broadcast channels, multiple-access channels, and source and channel coding with side information. The monograph concludes with a discussion of open problems and directions for future work. v vi Chapter 1 A: β: 0, c 2 , 0, c L , 0, , 0 0, M columns M columns M columns Section 1 Section 2 Section L T n rows 0, c 1 , 0, 0, Figure 1.1: A Gaussian sparse regression codebook of block length n: A is a design matrix with independent Gaussian entries, and β is a sparse vector with one non-zero in each of L sections. Codewords are of the form Aβ, i.e., linear combinations of the columns corresponding to the non-zeros in β. The message is indexed by the locations of the non-zeros, and the values c 1 , . . . , c L are fixed a priori.