Large Deviations Behavior of the Logarithmic Error Probability of Random Codes

Abstract

This work studies the deviations of the error exponent of the constant composition code ensemble around its expectation, known as the error exponent of the typical random code (TRC). In particular, it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the TRC exponent is exponentially small; upper and lower bounds for this exponent are given, which coincide in some cases. In addition, the probability of randomly drawing a codebook whose error exponent is larger than the TRC exponent is shown to be double-exponentially small; upper and lower bounds to the double-exponential exponent are given. The results suggest that codebooks whose error exponent is larger than the error exponent of the TRC are extremely rare. The key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables.

Concentration of Random-Coding Error Exponents

“…This way, the TRC emerges as the most likely error exponent in the random-coding ensemble as the block length n tends to infinity -if one wishes to improve the error exponent, one must improve the ensemble. The work in [11] shows such a concentration property by separately studying the the tails of the distribution of E n (C n ) for the constant composition ensemble and DMCs. It shows an interesting asymmetry: the probability P[E n (C n ) < E trc (R)] decays exponentially, while P[E n (C n ) > E trc (R)] decays double-exponentially.…”

“…This implies that, beyond the concentration property, it is significantly more difficult to find a code in the ensemble with exponent higher than E trc (R). This asymmetry is difficult to obtain from the proof of Theorem 1, as one would need to study separately the two tails, as done in [11].…”

“…random coding, the pairwise error probabilities P[X i → X j ], with, i = j are pairwise-independent random variables. Hence, the union bound P ub e (C n ) in ( 11) is a sum of M n pairwise-independent random variables, where concentration properties are expected to hold [11]. Let E ub trc (R) be the union-bound TRC exponent of (11), that is…”

“…Hence, the union bound P ub e (C n ) in ( 11) is a sum of M n pairwise-independent random variables, where concentration properties are expected to hold [11]. Let E ub trc (R) be the union-bound TRC exponent of (11), that is…”

“…Merhav derived the TRC exponent for spherical codes over colored Gaussian channels [9] and for random convolutional code ensembles [10]. The error exponent of a random fixed-composition code with GLD is known to converge in probability to the TRC [11], a convergence that is non-symmetric: the lower tail decays exponentially while the upper tail decays doubly-exponentially. The latter was first established for a limited range of rates in [12].…”