Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks

Abstract

Convolutional neural networks (CNNs) introduce stateof-the-art results for various tasks with the price of high computational demands. Inspired by the observation that spatial correlation exists in CNN output feature maps (ofms), we propose a method to dynamically predict whether ofm activations are zero-valued or not according to their neighboring activation values, thereby avoiding zerovalued activations and reducing the number of convolution operations. We implement the zero activation predictor (ZAP) with a lightweight CNN, which imposes negligible overheads and is easy to train and deploy on existing models. Furthermore, without model retraining, the same ZAP can be tuned to many different operating points along the accuracy-savings trade-off curve. For example, using VGG-16 and the ILSVRC-2012 dataset, two different operating points achieve a reduction of 20% and 30% multiplyaccumulate (MAC) operations with top-1/top-5 accuracy degradation of 0.1%/0.04% and 1.3%/0.7% without finetuning of the entire model, respectively. Considering oneepoch fine-tuning, 45% MAC operations may be reduced with 1.3%/0.7% accuracy degradation.

Anytime Dense Prediction with Confidence Adaptivity

“…Rather than choosing layers, spatial adaptivity chooses where to adjust the amount of computation across different spatial positions in the input. For example, the model could infer spatial masks for feature maps and skip computation on masked areas [47,12,33,44,7]. Figurnov et al [14] maintains a halting score at each pixel and once it reaches a threshold the model will stop inference at those positions for spatially coarse tasks like classification or bounding box detection.…”