Universal Decoding for the Typical Random Code and for the Expurgated Code

Tamir, Ran; Merhav, Neri

doi:10.1109/tit.2021.3137057

Cited by 7 publications

(10 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…which is the stochastic version of the well-known universal maximum mutual information (MMI) decoder [24], which has been recently proven to be universal in a typical error exponent sense [25]. The MMI decoder is approached by letting β → ∞ in (10) .…”

Section: Preliminariesmentioning

confidence: 99%

“…By studying the behavior of both tails, work in [15] proves concentration in probability. The TRC was shown to be universally achievable with a likelihood mutual-information decoder in [17]. For pairwise-independent ensembles and arbitrary channels, Cocco et al showed in [18] that the probability that a code in the ensemble has an exponent smaller than a lower bound on the TRC exponent is vanishingly small.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generalized Random Gilbert-Varshamov Codes: Typical Error Exponent and Concentration Properties

Truong,

Guillén i Fàbregas

2024

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We find the exact typical error exponent of constant composition generalized random Gilbert-Varshamov (RGV) codes over discrete memoryless channels with generalized likelihood decoding. We show that the typical error exponent of the RGV ensemble is equal to the expurgated error exponent, provided that the RGV codebook parameters are chosen appropriately. We also prove that the random coding exponent converges in probability to the typical error exponent, and the corresponding non-asymptotic concentration rates are derived. Our results show that the decay rate of the lower tail is exponential while that of the upper tail is double exponential above the expurgated error exponent. The explicit dependence of the decay rates on the RGV distance functions is characterized.

show abstract

Section: Preliminariesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generalized Random Gilbert-Varshamov Codes: Typical Error Exponent and Concentration Properties

Truong,

Guillén i Fàbregas

2024

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Later, Merhav has studied error exponents of TRCs for the colored Gaussian channel [22], typical random trellis codes [23], and has derived a Lagrange-dual lower bound to the TRC exponent [24]. Recently, Tamir et al have studied the large deviations behavior around the TRC exponent [35], error exponents of typical random Slepian-Wolf codes [33], and universal decoding for the TRC exponent [34]. More interesting concentration results of the logarithmic error probability of random codes around the error exponent of the TRC have been reported in [36].…”

Section: Introductionmentioning

confidence: 99%

“…For the above described code construction of variable-rate binning (that depends on the empirical distribution of the SI), we prove that at least for the random coding error exponent, the penalized MMI decoder is actually optimal among all metrics that depend both on the joint empirical distribution of the codeword and the channel output sequence as well as on the SI possible type 4 . Due to recent findings in [34],…”

Section: Introductionmentioning

confidence: 99%

Error Exponents of the Dirty-Paper and Gel'fand-Pinsker Channels

Tamir¹,

Merhav²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We derive various error exponents for communication channels with random states, which are available non-causally at the encoder only. For both the finite-alphabet Gel'fand-Pinsker channel and its Gaussian counterpart, the dirty-paper channel, we derive random coding exponents, error exponents of the typical random codes (TRCs), and error exponents of expurgated codes. For the two channel models, we analyze some sub-optimal bin-index decoders, which turn out to be asymptotically optimal, at least for the random coding error exponent. For the dirty-paper channel, we show explicitly via a numerical example, that both the error exponent of the TRC and the expurgated exponent strictly improve upon the random coding exponent, at relatively low coding rates, which is a known fact for discrete memoryless channels without random states. We also show that at rates below capacity, the optimal values of the dirty-paper design parameter α in the random coding sense and in the TRC exponent sense are different from one another, and they are both different from the optimal α that is required for attaining the channel capacity. For the Gel'fand-Pinsker channel, we allow for a variable-rate random binning code construction, and prove that the previously proposed maximum penalized mutual information decoder is asymptotically optimal within a given class of decoders, at least for the random coding error exponent.

show abstract

“…The concentration properties of the error exponent of randomly generated codes is studied in [14]. Tamir and Merhav [15] have shown that the typical error exponent can be achieved by a universal decoder ignorant of the channel law. Specifically, they have shown that a stochastic decoder based on the empirical mutual information between the received sequence and each codeword attains the typical error exponent.…”

mentioning

confidence: 99%

Typical Error Exponents: A Dual Domain Derivation

Cocco¹,

i²,

Font-Segura³

2022

Preprint

View full text Add to dashboard Cite

This paper shows that the probability that the error exponent of a given code randomly generated from a pairwiseindependent ensemble being smaller than a lower bound on the typical random-coding exponent tends to zero as the codeword length tends to infinity. This lower bound is known to be tight for i.i.d. ensembles over the binary symmetric channel and for constant-composition codes over memoryless channels. Our results recover both as special cases and remain valid for arbitrary alphabets, arbitrary channels-for example finite-state channels with memory-, and arbitrary pairwise-independent ensembles. We specialize our results to the i.i.d., constant-composition and costconstrained ensembles over discrete memoryless channels and to ensembles over finite-state channels. I. INTRODUCTIONSince Shannon's work [1], the random coding method remains a key tool in information theory and is applied to a wide variety of communication and compression settings. Specifically, by analysing the probability of error averaged over a certain ensemble of randomly generated codes, it is possible to show the existence of sequences of codes with a certain rate having vanishingly small probability of error.A refinement of the random coding analysis allows to show that, for any discrete memoryless channel (DMC), the ensemble average error probability decays exponentially according to a certain error exponent, often referred to as the random-coding exponent [2], [3]. The resulting exponent is known to be tight at high rates [4], but not at low rates. By expurgating poor codebooks from the ensemble, it is possible to show that there exist sequences of codes that attain the expurgated exponent, known to be strictly better than the random coding one at low rates [3], [5], [6]. While it is difficult to infer the structure of the resulting expurgated ensemble, it is useful to prove the existence of good low-rate codes. The expurgated exponent is known to be tight at rate that tends to zero [4].It is also possible to show the existence of codes that attain the maximum of the expurgated and random-coding exponents [6] and to construct explicit random-coding ensembles that attain the same maximum [7].

show abstract

Universal Decoding for the Typical Random Code and for the Expurgated Code

Cited by 7 publications

References 27 publications

Generalized Random Gilbert-Varshamov Codes: Typical Error Exponent and Concentration Properties

Generalized Random Gilbert-Varshamov Codes: Typical Error Exponent and Concentration Properties

Error Exponents of the Dirty-Paper and Gel'fand-Pinsker Channels

Typical Error Exponents: A Dual Domain Derivation

Contact Info

Product

Resources

About