On Distribution Preserving Quantization

Abstract

Upon compressing perceptually relevant signals, conventional quantization generally results in unnatural outcomes at low rates. We propose distribution preserving quantization (DPQ) to solve this problem.DPQ is a new quantization concept that confines the probability space of the reconstruction to be identical to that of the source. A distinctive feature of DPQ is that it facilitates a seamless transition between signal synthesis and quantization. A theoretical analysis of DPQ leads to a distribution preserving rate-distortion function (DP-RDF), which serves as a lower bound on the rate of any DPQ scheme, under a constraint on distortion. In general situations, the DP-RDF approaches the classic rate-distortion function for the same source and distortion measure, in the limit of an increasing rate. A practical DPQ scheme based on a multivariate transformation is also proposed. This scheme asymptotically achieves the DP-RDF for i.i.d. Gaussian sources and the mean squared error.

Randomized Quantization and Source Coding With Constrained Output Distribution

“…While for fixed quantizer dimension we can only provide existence results, for stationary and memoryless source and output distributions we also develop a rate-distortion theorem which identifies the minimum distortion in the limit of large block lengths in terms of the so-called output-constrained distortion-rate function. This last result solves a general version of a problem that was left open in [11].…”

“…The converse (Lemma 2 below) directly follows from the usual proof of the converse part of the rate-distortion theorem. In fact, this was first noticed in [11] where the special case ψ = µ was considered and (in a different formulation) it was shown that for all n…”

“…If we set ψ = µ, the above problem is reduced to showing the existence of a distribution-preserving randomized quantizer [10], [11] having minimum distortion.…”

“…Virtually the same argument implies that L n (µ, ψ, R) ≥ D(µ, ψ, R) for all n and ψ. Nevertheless, we write out the proof in Appendix H since, strictly speaking, the proof in [11] is only valid if ψ is discrete with finite (Shannon) entropy or it has a density and finite differential entropy. (b) The proof of the converse part (Lemma 2) is valid for any randomized quantizer whose output Y n satisfies Y i ∼ ψ, i = 1, .…”

“…Dithered quantizers or other structured randomized quantizers classes likely do not provide optimal performance in this problem. In an unpublished work [11] the same authors considered more general distribution-preserving randomized vector quantizers and lower bounded the minimum distortion achievable by such schemes when the source is stationary and memoryless.…”