Generative Low-bitwidth Data Free Quantization

Abstract

Neural network quantization is an effective way to compress deep models and improve the execution latency and energy efficiency, so that they can be deployed on mobile or embedded devices. Existing quantization methods require original data for calibration or fine-tuning to get better performance. However, in many real-world scenarios, the data may not be available due to confidential or private issues, making existing quantization methods not applicable. Moreover, due to the absence of original data, the recently developed generative adversarial networks (GANs) can not be applied to generate data. Although the full precision model may contain the entire data information, such information alone is hard to exploit for recovering the original data or generating new meaningful data. In this paper, we investigate a simple-yet-effective method called Generative Low-bitwidth Data Free Quantization to remove the data dependence burden. Specifically, we propose a Knowledge Matching Generator to produce meaningful fake data by exploiting classification boundary knowledge and distribution information in the pre-trained model. With the help of generated data, we are able to quantize a model by learning knowledge from the pre-trained model. Extensive experiments on three data sets demonstrate the effectiveness of our method. More critically, our method achieves much higher accuracy on 4-bit quantization than the existing data free quantization method.

Data-Free Neural Architecture Search via Recursive Label Calibration

“…Generative adversarial networks (GAN) [2,33] can generate images with high fidelity, but still need real images as a reference while training the generator, which is not fully data-free. Recently, Chen et al [5] and Xu et al [28] proposed to use a generator to synthesize images from a pre-trained model and simultaneously train the student network. Further, Yin et al [29] proposed to synthesize images from the pre-trained teacher network using regularization terms and Jensen-Shannon divergence loss.…”

Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization

“…DFC and ZeroQ proposed to synthesize calibration data by gradient descent under the supervision of BatchNorm statistics [10,3]. Generative Adversarial Networks (GAN) is also utilized to generate synthetic calibration data [37], in which the generator generates calibration data and the discriminator is the quantized model. One problem with these optimization-based methods is the high time complexity.…”

“…ZeroQ showed its arXiv:2105.07331v1 [cs.LG] 16 May 2021 effectiveness on various datasets including ImageNet [7] and MSCOCO [23]. Another method is to use Generative Adversarial Networks (GAN) to create fake calibration data [37]. The proposed framework named GDFQ is trained with BatchNorm statistics loss, Cross Entropy classification loss and knowledge distillation loss.…”