Polar coding for the Slepian-Wolf problem based on monotone chain rules

Abstract

Abstract-We give a polar coding scheme that achieves the full admissible rate region in the Slepian-Wolf problem without time-sharing. The method is based on a source polarization result using monotone chain rule expansions.Index Terms-Monotone chain rules, polar codes, Slepian-Wolf problem, source polarization. I. INTRODUCTIONConsider a memoryless source with generic variables (X, Y ) ∼ P X,Y where P X,Y is a fixed but arbitrary prob-. This paper considers the Slepian-Wolf problem for this source. As usual, the coding system consists of two encoders and one decoder. For a specified rate pair (R 1 , R 2 ), encoder 1 observes X N and encodes it into a codeword of length N R 1 bits; encoder 2 observes Y N and encodes it into a codeword of length N R 2 bits. The decoder observes the two codewords and is expected to recover (X N , Y N ) with small probability of error. The Slepian-Wolf result [1] states that this is possible if (R 1 , R 2 ) falls strictly inside the Slepian-Wolf rate region defined asThe subset of R SW consisting of points for which R x + R y = H(X, Y ) is referred to as the dominant face (of the rate region); and the points (R x , R y ) = (H(X), H(Y |X)) and (R x , R y ) = (H(X|Y ), H(Y )) are referred to as the corner points.Polar coding for the above Slepian-Wolf problem was first considered by Hussami et al [2] (see also Korada [3]) who showed that the corner points of R SW could be achieved by polar codes for the special case where P X and P Y are uniform on {0, 1}. In [4], this result was proved without any restrictions on P X and P Y . These results showed that polar codes could achieve the entire region R SW by time-sharing between two codes designed for the corner points.This paper is concerned with the question of whether polar codes can achieve R SW without aid from time-sharing. This question is motivated by the fact that there are random-coding methods, such as Cover's "binning" method [5], that do not require time-sharing to achieve R SW . Thus, the question is important for understanding the power of polar coding relative to other coding methods both as a proof method and also for practical applications.In fact, such questions on the relative power of polar coding first arose in the context of the multiple access channel (MAC), which is the dual of the Slepian-Wolf problem. In [6], Ş aşoglu

Polarization of the Rényi Information Dimension With Applications to Compressed Sensing

“…pI n , F n , Pq is an erasure stochastic process with initial value δ polarizing to t0, 1u.˝P roof: For n " 0, we have a Hadamard matrix of order N " 2 0 " 1 which is simply a number, thus, X 1 " Z 1 and we have I 0 p1q " dpZ 1 q " δ. Consider an arbitrary n, let N " 2 n and let I n be defined as in (34). We need to prove that I n satisfies the following recursion for i P r2 n s I n piq`" I n`1 p2i´1q " 2I n piq´I n piq 2 (35)…”

“…Theorem 7. pI n , F n , Pq and pJ n , F n , Pq are erasure processes with initial value dpU 1 q and dpV 1 |U 1 q respectively, polarizing to t0, 1u.P roof: Proof in Appendix D. By changing the order of expansion of dpX N 1 , Y N 1 q, i.e., first expanding with respect to Y N 1 and then with respect to X N 1 , we obtain another 2-dim erasure process pI n , J n q with the initial value pdpU 1 |V 1 q, dpV 1 qq, rather than pdpU 1 q, dpV 1 |U 1 qq. In fact, by applying the monotone chain rule expansion introduced in [34], we can expand dpX N 1 , Y N 1 q jointly (and simultaneously) in terms of Xs and Y s, thus, we can construct different 2-dim polarizing erasure processes pI n , J n q that converge almost surely to pI 8 , J 8 q P t0, 1u 2 . Also, the closure of the region of all possible pĪ,Jq :" ErpI 8 , J 8 qs for polarizing processes pI n , J n q contains the dominant face of the 2-dim region given by…”

“…, U d q, for some d ě 3. By applying the chain rule in different orders and using the results in [34], it is possible to build a d-dim erasure process pI…”

Codebook Cardinality Spectrum of Distributed Arithmetic Coding for Independent and Identically-Distributed Binary Sources

“…To uniquely indicate I(X n ), we need m ≥ n η(A, B) ≥ η(A, B) bits. Let us scale I(X n ) up by 2 n times to get the NMI N (X n ) of X n as defined by (5). Obviously,…”

“…Slepian-Wolf Coding (SWC) [1] is the lossless form of Distributed Source Coding (DSC) referring to separate compression and joint lossless reconstruction of two or more correlated discrete sources. Since the SWC can be mapped into a communication problem over a "virtual communication channel" [2], it is traditionally realized via channel codes, e.g., turbo codes [3], Low-Density Parity-Check (LDPC) codes [4], more recently polar codes [5], etc. Though channel coding has been recognized as a natural solution for the SWC problem, conventional source coding, e.g., Arithmetic Coding (AC) [6], [7], has also been tried.…”