Multiterminal source coding with high resolution

Abstract

We consider separate encoding and joint decoding of correlated continuous information sources, subject to a difference distortion measure. We first derive a multiterminal extension of the Shannon lower bound for the rate region. Then we show that this Shannon outer bound is asymptotically tight for small distortions. These results imply that the loss in the sum of the coding rates due to the separation of the encoders vanishes in the limit of high resolution. Furthermore, lattice quantizers followed by Slepian-Wolf lossless encoding are asymptotically optimal. We also investigate the high-resolution rate region in the remote coding case, where the encoders observe only noisy versions of the sources. For the quadratic Gaussian case, we establish a separation result to the effect that multiterminal coding aimed at reconstructing the noisy sources subject to the rate constraints, followed by estimation of the remote sources from these reconstructions, is optimal under certain regularity conditions on the structure of the coding scheme.

Distributed Scalar Quantization for Computing: High-Resolution Analysis and Extensions

“…This approach, with vector quantization performed on blocks of each of the sources, is optimal at all rates for jointly Gaussian sources and MSE distortion [4]. This approach is also optimal in the asymptotic regime of both large block length and high resolution [5]. The general lossy multiterminal source coding problem for large block length but finite rates, whether for discrete or continuous alphabet sources, is open.…”

High-rate quantization and transform coding with side information at the decoder

“…Later, an upper bound on the rate loss due to the unavailability of the side information at the encoder was found in [5], which also proved that for power-difference distortion measures and smooth source probability distributions, this rate loss vanishes in the limit of small distortion. A similar high-resolution result was obtained in [13] for distributed coding of several sources without side information, also from an informationtheoretic perspective, that is, for arbitrarily large dimension. In [14] (unpublished), it was shown that tessellating quantizers followed by SlepianWolf coders are asymptotically optimal in the limit of small distortion and large dimension.…”

“…The nondistributed case was studied in [40][41][42], and [7,[43][44][45]8] analyzed the distributed case from an information-theoretic point of view. Using Gaussian statistics and Mean-Squared Error (MSE) as a distortion measure, [13] proved that distributed coding of two noisy observations without side information can be carried out with a performance close to that of joint coding and denoising, in the limit of small distortion and large dimension. Most of the operational work on distributed coding of noisy sources, that is, for a fixed dimension, deals with quantization design for a variety of settings [46][47][48][49], but does not consider the characterization of such quantizers at high rates or transforms.…”

Asymptotic Design of Quantizers for Decentralized MMSE Estimation

“…Gish and Pierce [10] tell us that uniform quantizers followed by entropy encoders are nearly optimal in a singleterminal reproduction-oriented scenario, a (high-rate) result corroborated in [14] even in the vector quantization (VQ) case: quantization points form a lattice and even were collaborative quantization among the sources possible the optimal scheme would not use it. Placing this insight in a multiterminal setting, Zamir and Berger [32] show that uniform (actually lattice) quantizers followed by SW encoders are nearly optimal for reproduction purposes, in the high resolution regime 2 . The main question we address in this paper immediately arises:…”