Asymptotic Characteristics of MSE-Optimal Scalar Quantizers for Generalized Gamma Sources

Abstract

Characteristics, such as the support limit and distortions, of minimum mean-squared error (MSE) N-level uniform and nonuniform scalar quantizers are studied for the family of the generalized gamma density functions as N increases. For the study, MSE-optimal scalar quantizers are designed at integer rates from 1 to 16 bits/sample, and their characteristics are compared with corresponding asymptotic formulas. The results show that the support limit formulas are generally accurate. They also show that the distortion of nonuniform quantizers is observed to converge to the Panter-Dite asymptotic constant, whereas the distortion of uniform quantizers exhibits slow or even stagnant convergence to its corresponding Hui-Neuhoff asymptotic constant at the studied rate range, though it may stay at a close proximity to the asymptotic constant for the Rayleigh and Laplacian pdfs.Additional terms in the asymptote result in quite considerable accuracy improvement, making the formulas useful especially when rate is 8 or greater.

On the Characteristics of MSE-Optimal Symmetric Scalar Quantizers for the Generalized Gamma, Bucklew-Gallagher, and Hui-Neuhoff Sources

“…The formulas given in this section for the G-Γ pdf come directly from the previous studies [7,9,10] ; the formulas for the B-G and H-N densities are rederived [9] for additional terms.…”

“…To assess the accuracies of the formulas, optimal symmetric quantizers are designed and their characteristics are tabulated in Tables 3-8: uniform quantizers with integer rate  up to 20 bits are designed with direct minimization (so the rate is extended from 16 in the case of the common G-Γ pdfs [10] ); and nonuniform quantizers with up to 16 bits for the B-G pdf with the Lloyd-Max algorithm [11][12] and for the Hui-Neuhoff pdf up to 11 bits, beyond which the design has experienced a numerical challenge. Overall the designed quantizers agree with those [7] to the extent reported.…”

“…The principal results of the paper include asymptotic formulas for the characteristics that are newly derived herein for nonuniform quantizers for the Bucklew-Gallagher and Hui-Neuhoff pdfs, parallelling those for the generalized gamma pdfs [10] .…”

“…For the accuracy assessment of the formulas, optimal uniform quantizers are numerically designed using direct minimization for integer  up to 20 bits, extended from 16 bits [10] , and nonuniform quantizers using the Lloyd-Max algorithm [11][12] up to 16 except for the Hui-Neuhoff pdf, in which case the maximum rate is 11, beyond which the design is a challenge inviting much more care.…”

“…In general without actual design it is difficult to estimate accurately thresholds, quantization levels, and the mean-square error (MSE) distortion of an optimal (minimum MSE) quantizer. However, when the number of levels is large, the high resolution quantization theory [4][5][6][7][8][9][10] has discovered approximation formulas that are increasingly accurate with more number of levels for some of such characteristics. This paper extends in particular these two studies [9][10] , respectively, to nonuniform quantizers and to source probability density functions (pdfs) with heavier tails; investigates the uniqueness problem of the differentiation approach [9] for finding optimal step sizes; and assesses accuracies of these formulas by comparing them with those of numerically designed quantizers.…”