Proof of replica formulas in the high noise regime for communication using LDGM codes

Abstract

Abstract-We consider communication over a binary input memoryless output symmetric channel with low density generator matrix codes and optimal maximum a posteriori decoding. It is known that the problem of computing the average conditional entropy, over such code ensembles in the asymptotic limit of large block length, is closely related to computing the free energy of a mean field spin glass in the thermodynamic limit. Tentative explicit formulas for these quantities have been derived thanks to the replica method (of spin glass theory) and are generally conjectured to be exact. In this contribution we show that the replica solution is indeed exact in the high noise regime, where it coincides with density evolution equations. Our method uses ideas coming from high temperature expansions in spin glass theory. I. MOTIVATIONWe consider communication over a binary input memoryless output symmetric channel (BMS) with low density generator matrix (LDGM) codes and optimal maximum a posteriori (MAP) decoding. Let U 1 , U 2 , . . . , U n denote the information bits from which we create m generator bits X 1 , X 2 , . . . , X m using an LDGM code. The generator bits are transmitted through the channel with transition probability p Y |X and Y 1 , Y 2 , . . . , Y m is received. We are interested in computing the average (over the code ensemble) entropy of the information word U n given the received word Y m . The average over the code ensemble conditional entropyof the transmitted information word U n conditional to the received message Y m can be formally computed by the ill-defined replica method of statistical mechanics. From the rather explicit formula obtained in this way one may also compute its derivative with respect to the noise level, a quantity that is also called MAP GEXIT curve [1]. It is believed that replica symmetry breaking is absent for symmetric channels (for bit MAP decoding) and is conjectured that the replica symmetric equations are rigorously exact. It is well known that away from the intervals between BP and MAP thresholds, the replica formulas coincide with the ones given by density evolution. Thus the conjecture also tells that density evolution gives the exact conditional entropy and MAP GEXIT curve away from intervals separating BP and MAP thresholds While the general proof of this conjecture is still an open problem, some progress has been made in the last years. Tight bounds have been derived using a variety of tools (physical In this paper we provide a full proof of the conjecture in high noise regimes for the case of general LDGM code ensembles with bounded degrees and BIAWGN, BEC, BSC channels (and convex combinations of them). We believe that our proof can be extended to a more general class of BMS channels although some of the estimates become more technical. Apart from this result the interest of this work also lies in the method which departs from all the ones previously used. Our main tool is an expansion, that has its roots in high temperature expansions of statistical mechanics, and al...

Decay of correlations: An application to low-density parity check codes

“…However it is still unclear how to apply techniques of correlation decay to standard LDPC ensembles. In this work, we show that results of [8] can be applied to a special class of LDPC codes.…”

“…In this paragraph we reformulate the main results of [8] in a suitable way for the present analysis. It is convenient to define a general model on a bi-partite Tanner graph consisting as usual of variable nodes and check nodes.…”

“…The average of σ X = i∈X σ i with respect to µ(σ) is denoted as σ X . For later use, we explain a correlation decay lemma adapted from [8].…”

“…More recently a new method based on correlation decay techniques was put forward in [8] for the analysis of low density generator matrix codes (LDGM) and enabled to prove the correctness of the replica formulas in the high noise regime for a class of BMS channels. However it is still unclear how to apply techniques of correlation decay to standard LDPC ensembles.…”

“…

Abstract-Recently the decay of correlations between bits of low density generator matrix (LDGM) codes have been investigated by using high temperature expansions from statistical physics [8]. In this work we apply these ideas to a special class of low density parity check codes (LDPC) on the binary input gaussian white noise channel (BIAWGNC).

…”