Lossy Intermediate Deep Learning Feature Compression and Evaluation

Abstract

With the unprecedented success of deep learning in computer vision tasks, many cloud-based visual analysis applications are powered by deep learning models. However, the deep learning models are also characterized with high computational complexity and are task-specific, which may hinder the large-scale implementation of the conventional data communication paradigms. To enable a better balance among bandwidth usage, computational load and the generalization capability for cloud-end servers, we propose to compress and transmit intermediate deep learning features instead of visual signals and ultimately utilized features. The proposed strategy also provides a promising way for the standardization of deep feature coding. As the first attempt to this problem, we present a lossy compression framework and evaluation metrics for intermediate deep feature compression. Comprehensive experimental results show the effectiveness of our proposed methods and the feasibility of the proposed data transmission strategy. It is worth mentioning that the proposed compression framework and evaluation metrics have been adopted into the ongoing AVS (Audio Video Coding Standard Workgroup) -Visual Feature Coding Standard.

Auto-Tiler: Variable-Dimension Autoencoder with Tiling for Compressing Intermediate Feature Space of Deep Neural Networks for Internet of Things

“…There are researches to compress intermediate features in collaborative intelligence in order to reduce the transfer time [ 13 , 14 , 15 , 16 ]. Some suggest preprocessing methods to make feature space easily compressed by conventional image/video codecs such as JPEG and HEVC.…”

“…These codecs have evolved over decades and can achieve a very high compression ratio on vision data. Compressing intermediate features is relatively new and many studies [ 13 , 14 , 15 , 16 ] extend existing image/video codecs to compress vision-based intermediate features. They add a preprocessing stage before applying an image/video codec to fit intermediate features into the target codec.…”

“…In most convolutional neural networks (CNN), channel count increases and feature map size decreases as layers deepen. Some studies [ 13 , 14 ] individually apply a codec to these many small-sized maps, which suffers from limited redundancy and unamortized header costs. The others [ 15 , 16 ] tile multiple maps to build a large frame, which introduces blockiness in the combined frame and degrades the efficiency of natural-image-based codecs.…”

Bit Allocation for Multi-Task Collaborative Intelligence

“…Authors in [6,24] describe general lossy and lossless codecs for deep feature compression. Their focus is on the features of popular DNN backbones rather than task-specific features.…”

“…Due to their focus on single tensor compression, none of the studies mentioned above consider optimal bit allocation to multiple tensors. Even in [6,24] where compression of multiple features is considered, the compression is performed without joint bit allocation. The main contribution of the present paper are the solutions to bit allocation problems in several multi-stream CI scenarios.…”