The Dispersion of Lossy Source Coding

Abstract

In this work we investigate the behavior of the minimal rate needed in order to guarantee a given probability that the distortion exceeds a prescribed threshold, at some fixed finite quantization block length. We show that the excess coding rate above the rate-distortion function is inversely proportional (to the first order) to the square root of the block length. We give an explicit expression for the proportion constant, which is given by the inverse Q-function of the allowed excess distortion probability, times the square root of a constant, termed the excess distortion dispersion. This result is the dual of a corresponding channel coding result, where the dispersion above is the dual of the channel dispersion. The work treats discrete memoryless sources, as well as the quadratic-Gaussian case.

Nonasymptotic and Second-Order Achievability Bounds for Coding With Side-Information

“…Subsequently we find an upper bound to the (n, ε)-Wyner-Ziv rate-distortion function R WZ (n, ε) defined in (21). We also show that the (direct part of the) dispersion of lossy source coding found by Ingber-Kochman [25] and Kostina-Verdú [26] can be recovered from the CS-type bound in Corollary 9. This is not unexpected because the lossy source coding (rate-distortion) problem is a special case of the WynerZiv problem where the side-information is absent.…”

“…3] because the error events are under the same error probability. Notice that unlike the existing asymptotic and non-asymptotic results for GP coding [6], [10], [49], the channel input x satisfies the cost constraint (25) or its almost sure equivalent (cf. Proposition 1).…”

“…In addition, since the canonical ratedistortion problem [27] is a special case of the WZ problem, we show that our non-asymptotic achievability bound for the WZ problem, when suitably specialized, yields the correct dispersion for lossy source coding [25], [26]. We do so using two methods: (i) the method of types [28] and (ii) results involving the D-tilted information [26].…”

“…If the cost constraint (25) is absent (i.e., every codeword in X n is admissible), we will write C GP (n, ε) instead of C GP (n, ε, ∞), P e (Φ n ) instead of P e (Φ n ; ∞) and so on.…”

“…Since the lossy source coding problem is a special case of WZ coding, we use a specialization of the bound in (67) to derive an achievable dispersion (or second-order coding rate) of lossy source coding [25], [26], which turns out to be tight.…”