The Transmission Distortion of a Source as a Function of the Encoding Block Length*

Abstract-We characterize the rate distortion function for the source coding with decoder side information setting when the ith reconstruction symbol is allowed to depend only on the first i +s ide information symbols, for some finite look-ahead`, in addition to the index from the encoder. For the case of causal side information, i.e.,`= 0, we find that the penalty of causality is the omission of the subtracted mutual information term in the Wyner-Ziv rate distortion function. For`> 0, we derive a computable "infinite-letter" expression for the rate distortion function. When specialized to the near-lossless case, our results characterize the best achievable rate for the Slepian-Wolf source coding problem with finite side information looka-head, and have some surprising implications. We find that side information is useless for any fixedẁ hen the joint probability mass function (PMF) of the source and side information satisfies the positivity condition P (x; y) > 0 for all (x; y). More generally, the optimal rate depends on the distribution of the pair X; Y only through the distribution of X and the bipartite graph whose edges represent the pairs x; y for which P (x; y) > 0. On the other hand, if side information look-ahead is allowed to grow faster than logarithmic in the block length, then H (X j Y ) is achievable. Finally, we apply our approach to derive a computable expression for channel capacity when state information is available at the encoder with limited look-ahead.

Section: Discussionmentioning

confidence: 99%

“…3 Let be a partition of into the vertex sets associated with maximally connected components of , and let denote the index of the set for which . As in [39], we define the common information random variable as (27) Note that for every such that both and . Thus, is a deterministic function both of and of .…”

Section: Alternative Characterization Ofmentioning

confidence: 99%

Source Coding With Limited-Look-Ahead Side Information at the Decoder

Weissman

Gamal

2006

“…For discrete source-channel pairs with average distortion criterion, Pilc's achievability bound [12], [13] applies. For the transmission of a Gaussian source over a discrete channel under the average mean square error constraint, Wyner's achievability bound [14], [15] applies.…”

Section: Introductionmentioning

confidence: 99%

“…The distortion-dispersion function of joint source-channel coding is defined as If there is no cost constraint, we will simplify notation by dropping α from (5), (6), (7), (8), (11) and (12).…”

mentioning

confidence: 99%

Lossy Joint Source-Channel Coding in the Finite Blocklength Regime

Kostina

Verdú

2013

142

107

Abstract-This paper finds new tight finite-blocklength bounds for the best achievable lossy joint source-channel code rate, and demonstrates that joint source-channel code design brings considerable performance advantage over a separate one in the non-asymptotic regime. A joint source-channel code maps a block of k source symbols onto a length−n channel codeword, and the fidelity of reproduction at the receiver end is measured by the probability ǫ that the distortion exceeds a given threshold d. For memoryless sources and channels, it is demonstrated that the parameters of the best joint source-channel code must satisfy, where C and V are the channel capacity and channel dispersion, respectively; R(d) and V(d) are the source rate-distortion and rate-dispersion functions; and Q is the standard Gaussian complementary cdf. Symbol-bysymbol (uncoded) transmission is known to achieve the Shannon limit when the source and channel satisfy a certain probabilistic matching condition. In this paper we show that even when this condition is not satisfied, symbol-by-symbol transmission is, in some cases, the best known strategy in the non-asymptotic regime.

“…In Section IV of the above paper, Yu and Speed conjecture that n-l log n is indeed the best attainable convergence rate and this belief stems from a known lower bound on the rate-distortion performance (as a function of n) due to Pilc [2]. However, Pilc's lower bound is stated for the dual problem, i.e., fixed coding rate and minimum expected distortion.…”

Section: Introductionmentioning

confidence: 99%

A comment on "A rate of convergence result for a universal D-semifaithful code"

Merhav

1995

argument, that the expected compression ratio is less thanNeri Merhav, Senior Member, IEEE Absfract-In the above paper,' Yu and Speed propose a universal pointwise D-semifaithfnl code whose expected compression ratio, for discrete memoryless sources, approaches the rate-distortion function at a rate O(n-'logn). They also conjecture that this is the fastest achievable convergence rate for pointwise D-semifaithful codes. In this correspondence, we use a simple extension of Kraft's inequality and prove that this conjecture is true, at least for the Hamming distortion measure.