Design of a 2-Bit Neural Network Quantizer for Laplacian Source

Abstract

Achieving real-time inference is one of the major issues in contemporary neural network applications, as complex algorithms are frequently being deployed to mobile devices that have constrained storage and computing power. Moving from a full-precision neural network model to a lower representation by applying quantization techniques is a popular approach to facilitate this issue. Here, we analyze in detail and design a 2-bit uniform quantization model for Laplacian source due to its significance in terms of implementation simplicity, which further leads to a shorter processing time and faster inference. The results show that it is possible to achieve high classification accuracy (more than 96% in the case of MLP and more than 98% in the case of CNN) by implementing the proposed model, which is competitive to the performance of the other quantization solutions with almost optimal precision.

Whether the Support Region of Three-Bit Uniform Quantizer Has a Strong Impact on Post-Training Quantization for MNIST Dataset?

“…Namely, NN normalization methods have already been confirmed to work well empirically [ 37 ], whereby a lot of them implicitly assume that the distributions of normalized NN parameters, primarily weights, initially have zero mean and unit variance. Keeping in mind that the weights’ distribution can closely fit some well-known probability density functions, such as the Laplacian probability density function (pdf) is [ 2 ], in this paper, as in [ 12 , 13 , 34 , 38 , 39 , 40 ], we assume the Laplacian-like distribution for experimental weights’ distribution and the Laplacian pdf for the theoretical distribution of weights to estimate the performance of the three-bit uniform quantizer (three-bit UQ) in question. Our motivation to address the simplest UQ also stems from the fact that UQs are not optimal for nonlinear distributions.…”

“…For symmetrical data distributions, symmetrical quantizers with an even number of quantization steps are preferable [ 14 , 24 , 25 , 26 , 27 , 28 , 29 , 32 , 34 , 35 , 38 , 40 ]. Since it can be expected that most of the real data is asymmetrical, one could not conjecture that the preferable quantizer is also asymmetrical.…”

“…As already mentioned, the weight distribution can closely fit some well-known pdfs, such as Laplacian pdf is [ 2 ]. Accordingly, as in [ 12 , 13 , 34 , 38 , 39 , 40 ], we assume the Laplacian-like distribution for the experimental distribution of weights and the Laplacian pdf for the theoretical distribution of weights to estimate the performance of our three-bit UQ in question.…”

“…Similarly, as in numerous papers about quantization (for instance, in [ 6 , 9 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 32 , 34 , 35 , 36 , 38 , 40 ]), we are interested in conducting an analysis for the variance-matched case, where the designed for and applied to variance of data being quantized match and commonly amount to 1. Therefore, we further assume σ 2 = 1 so that we end up with the following novel closed-form formula for the distortion of the symmetrical three-bit UQ when the input has the Laplacian pdf of zero mean and unit variance: Let us finally define the theoretical SQNR as: which will also be calculated for σ 2 = 1 and compared with the experimentally determined SQNR ex UQ .…”

Iterative Algorithm for Parameterization of Two-Region Piecewise Uniform Quantizer for the Laplacian Source

“…It is well known that a nonuniform quantizer model, well accommodated to the signal's amplitude dynamic and a nonuniform pdf, has lower quantization error compared to the uniform quantizer (UQ) model with an equal number of quantization levels or equal bit-rates [2,11,13,18,[20][21][22][23][24][25][26][27]. However, due to the fact that UQ is the simplest quantizer model, it has been intensively studied, for instance in [23,24,[28][29][30][31][32].…”