Sparse Regression Codes

Abstract

Developing computationally-efficient codes that approach the Shannon-theoretic limits for communication and compression has long been one of the major goals of information and coding theory. There have been significant advances towards this goal in the last couple of decades, with the emergence of turbo codes, sparse-graph codes, and polar codes. These codes are designed primarily for discrete-alphabet channels and sources. For Gaussian channels and sources, where the alphabet is inherently continuous, Sparse Superposition Codes or Sparse Regression Codes (SPARCs) are a promising class of codes for achieving the Shannon limits.This monograph provides a unified and comprehensive over-view of sparse regression codes, covering theory, algorithms, and practical implementation aspects. The first part of the monograph focuses on SPARCs for AWGN channel coding, and the second part on SPARCs for lossy compression (with squared error distortion criterion). In the third part, SPARCs are used to construct codes for Gaussian multi-terminal channel and source coding models such as broadcast channels, multiple-access channels, and source and channel coding with side information. The monograph concludes with a discussion of open problems and directions for future work. v vi Chapter 1 A: β: 0, c 2 , 0, c L , 0, , 0 0, M columns M columns M columns Section 1 Section 2 Section L T n rows 0, c 1 , 0, 0, Figure 1.1: A Gaussian sparse regression codebook of block length n: A is a design matrix with independent Gaussian entries, and β is a sparse vector with one non-zero in each of L sections. Codewords are of the form Aβ, i.e., linear combinations of the columns corresponding to the non-zeros in β. The message is indexed by the locations of the non-zeros, and the values c 1 , . . . , c L are fixed a priori.

A Hybrid Approach to Coded Compressed Sensing Where Coupling Takes Place Via the Outer Code

“…In the next step, the value of every variable node is translated into an index representation of length m. This action is emblematic of CCS [3], and it produces a one-sparse block. The index vectors from the L variable nodes are then aggregated into vector m, which possesses a structure reminiscent of sparse regression codes [11], [15]. The transmitted codeword is obtained by multiplying m by a judiciously designed matrix AD, i.e., ADm is transmitted over the channel.…”

“…This framework is rooted in a divide-and-conquer approach where support recovery is broken down into several sub-problems, each of a size amenable to the application of standard CS solvers, such as non-negative least squares (NNLS) or approximate message passing (AMP). This reduction is enabled through an architecture that contains a concatenated code structure reminiscent of sparse regression codes [11] and for-all sparse recovery [12]. Once fragments are obtained by the CS solvers, they are stitched together using the outer tree code, yielding the desired support of the sparse vector.…”

Multi-Class Unsourced Random Access via Coded Demixing

“…Every output symbol is turned into an index vector, which contains zeros everywhere except for a single location where it features a one. These index vectors are concatenated into a SPARC-like vector [23]- [25], which is subsequently multiplied by a sensing matrix. Such a construction is somewhat intricate, and it can hardly be explained in detail within a short article.…”

Stochastic Binning and Coded Demixing for Unsourced Random Access

“…The conceptual starting point for our discussion is the CCS scheme of Amalladinne et al [6], [13]. This URA approach combines an LDPC outer code and a CS-style inner code reminiscent of a sparse regression code (SPARC) [14], [15]. More formally, an information message w is encoded using a non-binary LDPC code, yielding codeword…”