The dispersion of joint source-channel coding

We derive the optimal second-order coding region and moderate deviations constant for successive refinement source coding with a joint excess-distortion probability constraint. We consider two scenarios: (i) a discrete memoryless source (DMS) and arbitrary distortion measures at the decoders and (ii) a Gaussian memoryless source (GMS) and quadratic distortion measures at the decoders. For a DMS with arbitrary distortion measures, we prove an achievable second-order coding region, using type covering lemmas by Kanlis and Narayan and by No, Ingber and Weissman. We prove the converse using the perturbation approach by Gu and Effros. When the DMS is successively refinable, the expressions for the second-order coding region and the moderate deviations constant are simplified and easily computable. For this case, we also obtain new insights on the second-order behavior compared to the scenario where separate excess-distortion proabilities are considered. For example, we describe a DMS, for which the optimal second-order region transitions from being characterizable by a bivariate Gaussian to a univariate Gaussian, as the distortion levels are varied. We then consider a GMS with quadratic distortion measures. To prove the direct part, we make use of the sphere covering theorem by Verger-Gaugry, together with appropriately-defined Gaussian type classes. To prove the converse, we generalize Kostina and Verdú's one-shot converse bound for point-to-point lossy source coding. We remark that this proof is applicable to general successively refinable sources. In the proofs of the moderate deviations results for both scenarios, we follow a strategy similar to that for the second-order asymptotics and use the moderate deviations principle.

Section: B General Discrete Memoryless Sourcesmentioning

confidence: 53%

Second-Order and Moderate Deviation Asymptotics for Successive Refinement

Zhou

Tan

Motani

2017

“…Non-asymptotic achievability and converse bounds for a graph-theoretic model of JSCC have been obtained by Csiszár [16]. Most recently, Tauste Campo et al [17] showed a number of finite-blocklength random-coding bounds applicable to the almost-lossless JSCC setup, while Wang et al [18] found the dispersion of JSCC for sources and channels with finite alphabets.…”

Section: Introductionmentioning

confidence: 99%

Lossy Joint Source-Channel Coding in the Finite Blocklength Regime

Kostina

Verdú

2013

142

107

Abstract-This paper finds new tight finite-blocklength bounds for the best achievable lossy joint source-channel code rate, and demonstrates that joint source-channel code design brings considerable performance advantage over a separate one in the non-asymptotic regime. A joint source-channel code maps a block of k source symbols onto a length−n channel codeword, and the fidelity of reproduction at the receiver end is measured by the probability ǫ that the distortion exceeds a given threshold d. For memoryless sources and channels, it is demonstrated that the parameters of the best joint source-channel code must satisfy, where C and V are the channel capacity and channel dispersion, respectively; R(d) and V(d) are the source rate-distortion and rate-dispersion functions; and Q is the standard Gaussian complementary cdf. Symbol-bysymbol (uncoded) transmission is known to achieve the Shannon limit when the source and channel satisfy a certain probabilistic matching condition. In this paper we show that even when this condition is not satisfied, symbol-by-symbol transmission is, in some cases, the best known strategy in the non-asymptotic regime.

“…The direct part of the proof of the theorem in the original Wyner-Ziv paper [4] is based on the average fidelity criterion in (24). It relies on the compress-bin idea.…”

Section: B First-order Results For the Wz Problemmentioning

confidence: 99%

“…The pre-factor of this term S (V, ε), is likened to the dispersion [22], [24]- [26], and depends not only the variances of the information and entropy densities but also their correlations.…”

Section: A Main Contributionsmentioning

confidence: 99%

Nonasymptotic and Second-Order Achievability Bounds for Coding With Side-Information

Watanabe

Kuzuoka

Tan

2015

Abstract-We present novel non-asymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include (i) the WynerAhlswede-Körner (WAK) problem of almost-lossless source coding with rate-limited side-information, (ii) the Wyner-Ziv (WZ) problem of lossy source coding with side-information at the decoder and (iii) the Gel'fand-Pinsker (GP) problem of channel coding with noncausal state information available at the encoder. The bounds are proved using ideas from channel simulation and channel resolvability. Our bounds for all three problems improve on all previous non-asymptotic bounds on the error probability of the WAK, WZ and GP problems-in particular those derived by Verdú. Using our novel non-asymptotic bounds, we recover the general formulas for the optimal rates of these side-information problems. Finally, we also present achievable second-order coding rates by applying the multidimensional Berry-Esséen theorem to our new non-asymptotic bounds. Numerical results show that the second-order coding rates obtained using our non-asymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds.