Turbo codes: the phase transition

Montanari, Andrea

doi:10.1007/s100510070085

Cited by 72 publications

(30 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, there is a well-understood connection between statistical mechanics and belief propagation-based decoding of error correcting codes [6], [7]. These connections may suggest improved iterative algorithms for sparse estimation as well.…”

Section: Discussionmentioning

confidence: 99%

“…Indeed, the replica method and related ideas from statistical mechanics have found success in a number of classic NP-hard problems including the traveling salesman problem [2], graph partitioning [3], k-SAT [4] and others [5]. Statistical physics methods have also been applied to the study of error correcting codes [6], [7]. There are now several general texts on the replica method [8]- [11].…”

Section: A Replica Methods and Contributions Of This Workmentioning

confidence: 99%

See 1 more Smart Citation

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Rangan

Fletcher

Goyal

2012

IEEE Trans. Inform. Theory

173

298

View full text Add to dashboard Cite

Abstract-The replica method is a non-rigorous but wellknown technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verdú.The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hardthreshold. Among other benefits, the replica method provides a computationally-tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: A Replica Methods and Contributions Of This Workmentioning

confidence: 99%

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Rangan

Fletcher

Goyal

2012

IEEE Trans. Inform. Theory

173

298

View full text Add to dashboard Cite

show abstract

“…n c is reached by the optimum (but unknown) decoder. Turbo Codes also have been analysed using statistical mechanics [19,20]. Turbo Codes are based on recursive convolutional codes.…”

mentioning

confidence: 99%

“…Two short range one dimensional chains are coupled through the infinite range, non local constraint, Equ. (20). This constraint is non local because neighboring i's are not mapped to neighboring j's under the random permutation.…”

mentioning

confidence: 99%

“…The statistical mechanical models however, are completely different. Let me also mention that, under some reasonable assumptions, the iterative decoding algorithm for turbo codes (turbodecoding algorithm), which I am not explaining here, can be viewed [20] as a time discretisation of the Kolmogorov, Petrovsky and Piscounov equation [21]. This KPP equation has traveling wave solutions.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Statistical mechanics and capacity-approaching error-correcting codes

Sourlas

2001

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

I will show that there is a deep relation between error-correction codes and certain mathematical models of spin glasses. In particular minimum error probability decoding is equivalent to finding the ground state of the corresponding spin system. The most probable value of a symbol is related to the magnetization at a different temperature. Convolutional codes correspond to one-dimensional spin systems and Viterbi's decoding algorithm to the transfer matrix algorithm of Statistical Mechanics.I will also show how the recently discovered (or rediscovered) capacity approaching codes (turbo codes and low density parity check codes) can be analysed using statistical mechanics. It is possible to show, using statistical mechanics, that these codes allow error-free communication for signal to noise ratio above a certain threshold. This threshold depends on the particular code, and can be computed analytically in many cases. LPTENS 01/ 31It has been known[1 − 4] that error-correcting codes are mathematically equivalent to some theoretical spin-glass models. As it is explained in Forney's paper in this volume, there have been recently very interesting new developments in error-correcting codes. It is now possible to approach practically very close to Shannon's channel capacity. First came the discovery of turbo codes by Berrou and Glavieux [5] and later the rediscovery of low density parity check codes [6], first discovered by Gallager [7,8], in his thesis, in 1962. Both turbo codes and low density parity check (LPDC) codes are based on random structures. It turns out, as I will explain later, that it is possible to use their equivalence with spin glasses, to analyse them using the methods of statistical mechanics.Let me start by fixing the notations. Each information message consists of a sequence of K bits u = {u 1 , · · · , u K }, u i = 0 or 1. The binary vector u is called the source-word. Encoding introduces redundancy into the message. One maps u → x by encoding. u → x has to be a one to one map for the code to be meaningful. The binary vector x has N > K components. It is called a code-word. The ratio R = K/N which specifies the redundancy of the code, is called the rate of the code. One particularly important family of codes are the so-called linear codes. Linear codes are defined by x = G u G is a binary (i.e; its elements are zero or one) (N × K) matrix and the multiplication is modulo two. G is called the generating matrix of the code. Obviously by construction all the components x i of a code-word x are not independent. Of all the 2 N binary vectors only 2 K = 2 N R , those corresponding to a vector u, are code-words. Codewords satisfy the linear constraints (called parity check constraints) H x = 0 (modulo two), where H is a (K ×N ) binary matrix, called the parity check

show abstract

Generic Multiuser Detection and Statistical Physics

Guo

Tanaka

2008

Advances in Multiuser Detection

View full text Add to dashboard Cite

Turbo codes: the phase transition

Cited by 72 publications

References 23 publications

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Statistical mechanics and capacity-approaching error-correcting codes

Generic Multiuser Detection and Statistical Physics

Contact Info

Product

Resources

About