Third-Order Asymptotics of Variable-Length Compression Allowing Errors

Abstract

This study investigates the fundamental limits of variable-length compression in which prefix-free constraints are not imposed (i.e., one-to-one codes are studied) and non-vanishing error probabilities are permitted. Due in part to a crucial relation between the variable-length and fixed-length compression problems, our analysis requires a careful and refined analysis of the fundamental limits of fixed-length compression in the setting where the error probabilities are allowed to approach either zero or one polynomially in the blocklength. To obtain the refinements, we employ tools from moderate deviations and strong large deviations. Finally, we provide the third-order asymptotics for the problem of variable-length compression with non-vanishing error probabilities. We show that unlike several other informationtheoretic problems in which the third-order asymptotics are known, for the problem of interest here, the third-order term depends on the permissible error probability.Index Terms-Variable-length compression, Third-order asymptotics, Average codeword lengths, Moderate deviations, Cramér-type large deviations, Strong large deviations *as → ∞ for fixed 0 ≤ ≤ 1, where prefix ( | ) denotes the minimum of average codeword lengths of binary prefixfree codes for i.i.d. copies of in which the error

On Finite Blocklength Lossy Source Coding

“…The result of Han was later generalized by Koga and Yamamoto [92], who showed that the asymptotic average codeword length per source symbol of an optimal code is less than the source entropy if a non-vanishing error probability is tolerated, demonstrating the asymptotic advantage of variable length compression allowing errors. The result in [92] was refined by Kostina, Polyanskiy and Verdú [18] who derived the second-order asymptotic approximation and further refined by Sakai, Yavas, Tan [93] who further derived the third order asymptotic approximation. The above studies were also generalized to the case with side information [94,95].…”

“…This monograph focused on fixed-length lossy source coding. Motivated by the need to reduce the codeword length of frequently appeared symbols, fixedto-variable length (FVL) source coding has also been widely studied for the point-to-point case [91,92,18,136,137,93]. In particular, Kostina and Verdú derived the second-order asymptotics for average codeword length of the FVL rate-distortion problem subject to a non-vanishing excess-distortion probability, which was presented in Chapter 8.…”

Common Information, Noise Stability, and Their Extensions

“…Hence, the strong converse holds for Gács and Körner's formulation, while it does not hold for Csiszár and Narayan's formulation (i.e., Definition 3.1.2). This observation resembles lossless source coding in that the strong converse holds for the fixed-length version [180] but not the weak variable-length version [72], [98], [100], [146].…”