Improved Source Coding Exponents via Witsenhausen's Rate

2015 IEEE International Symposium on Information Theory (ISIT)

2015

We analyze the optimal trade-off between the error exponent and the excess-rate exponent for variable-rate Slepian-Wolf codes. In particular, we first derive upper (converse) bounds on the optimal error and excess-rate exponents, and then lower (achievable) bounds, via a simple class of variable-rate codes which assign the same rate to all source blocks of the same type class. Then, using the exponent bounds, we derive bounds on the optimal rate functions, namely, the minimal rate assigned to each type class, needed in order to achieve a given target error exponent. The resulting excess-rate exponent is then evaluated. Iterative algorithms are provided for the computation of both bounds on the optimal rate functions and their excess-rate exponents. The resulting Slepian-Wolf codes bridge between the two extremes of fixed-rate coding, which has minimal error exponent and maximal excess-rate exponent, and average-rate coding, which has maximal error exponent and minimal excess-rate exponent. Index TermsSlepian-Wolf coding, variable-rate coding, buffer overflow, excess-rate exponent, error exponent, reliability function, random-binning, alternating minimization.for the analysis of SW codes. In Section III, we derive upper and lower bounds on the error exponent and excessrate exponent of general SW codes, and discuss the trade-off between the two exponents. Then, in Section IV, we characterize the optimal rate allocation (in a sense that will be made precise), under an error exponent constraint, and in Section V, we analyze the resulting excess-rate exponent. In Section VI, we discuss computational aspects of the bounds on the optimal rate allocation, as well as the bounds on the optimal excess-rate exponent. Section VII demonstrates the results via a numerical example, and Section VIII summarizes the paper, along with directions for further research. Almost all proofs are deferred to Appendix A. In Appendix B, we provide several general results on the reliability function of channel coding, which are required in order to fully understand the proofs in Appendix A. In Appendix C and Appendix D, we provide some side results, and Appendix E we provide some useful Lemmas. 4

Section: Proof Of Lemma 26mentioning

confidence: 99%

Optimum trade-offs between error exponent and excess-rate exponent of Slepian-Wolf coding

Weinberger

2015 IEEE International Symposium on Information Theory (ISIT)

2015

“…More recently, Csiszár [1] and Oohama and Han [13] have derived error exponents for the more general setting of coded side information. For large rates at one of the encoders, Kelly and Wagner [10] improved upon these results, but they did not consider the general case.…”

Section: Introductionmentioning

confidence: 99%

Erasure/List Exponents for Slepian–Wolf Decoding

IEEE Trans. Inform. Theory

2014

We analyze random coding error exponents associated with erasure/list Slepian-Wolf decoding using two different methods and then compare the resulting bounds. The first method follows the well known techniques of Gallager and Forney and the second method is based on a technique of distance enumeration, or more generally, type class enumeration, which is rooted in the statistical mechanics of a disordered system that is related to the random energy model (REM). The second method is guaranteed to yield exponent functions which are at least as tight as those of the first method, and it is demonstrated that for certain combinations of coding rates and thresholds, the bounds of the second method are strictly tighter than those of the first method, by an arbitrarily large factor. In fact, the second method may even yield an infinite exponent at regions where the first method gives finite values. We also discuss the option of variable-rate Slepian-Wolf encoding and demonstrate how it can improve on the resulting exponents.

“…For example, it was shown in [22], [27], and [38] that variablerate SW codes might have lower redundancy (additional rate beyond the conditional entropy, for a given error probability). Other results on variable-rate coding can be found in [26], [28], and [33]. In another line of work, which is more relevant to this paper, it was observed that variable-rate coding under an average rate constraint [6], [7], [9] outperforms fixedrate coding in terms of error exponents.…”

mentioning

confidence: 60%

Optimum Tradeoffs Between the Error Exponent and the Excess-Rate Exponent of Variable-Rate Slepian–Wolf Coding

Weinberger

IEEE Trans. Inform. Theory

2015

We analyze the optimal tradeoff between the error exponent and the excess-rate exponent for variable-rate Slepian-Wolf codes. In particular, we first derive upper (converse) bounds on the optimal error and excess-rate exponents, and then lower (achievable) bounds, via a simple class of variable-rate codes which assign the same rate to all source blocks of the same type class. Then, using the exponent bounds, we derive bounds on the optimal rate functions, namely, the minimal rate assigned to each type class, needed in order to achieve a given target error exponent. The resulting excess-rate exponent is then evaluated. Iterative algorithms are provided for the computation of both bounds on the optimal rate functions and their excess-rate exponents. The resulting Slepian-Wolf codes bridge between the two extremes of fixed-rate coding, which has minimal error exponent and maximal excess-rate exponent, and averagerate coding, which has maximal error exponent and minimal excess-rate exponent.