Deep Feature Compression Using Spatio-Temporal Arrangement Toward Collaborative Intelligent World

Suzuki, Satoshi; Takeda, Shoichiro; Takagi, Motohiro; Tanida, Ryuichi; Kimata, Hideaki; Shouno, Hayaru

doi:10.1109/tcsvt.2021.3107716

Cited by 16 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, the extracted features are encoded as bitstreams and sent to the decoder for image reconstruction. Moreover, all feature maps are spliced into one feature map, 28 and the following codec codes it into bitstreams. In fact, the following feature compression process involves feature quantization and feature coding.…”

Section: Proposed Methodsmentioning

confidence: 99%

Deep feature extraction and compression for human–machine vision

Huang

Yang

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

Latent representation features in deep learning (DL) exhibit excellent potential for visual data applications. For example, in traffic monitoring and video surveillance, the features simultaneously perform image analysis for machine vision and image reconstruction for human viewing. However, the existing deep features that appeal to machine and human receivers are always combinations of separated pieces and specific features. Due to these features being extracted from different branches in collaboration frameworks, the inherent relations between machine and human vision are insufficiently explored. Therefore, to obtain one set of representative and generic features, we propose a dynamic groupwise splitting network based on image content to explore and extract generic features for the two different receivers. First, we analyze the characteristics of the latent features and adopt intermediate features as the base features. Then, a feature classification and transformation mechanism based on image content is proposed to enhance the base features for further image reconstruction and analysis. Consequently, an end-to-end model with multimodel cascading and multistage training realizes both machine and human vision tasks. Extensive experiments show that our human-machine vision collaboration framework has high practical value and performance.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Deep feature extraction and compression for human–machine vision

Huang

Yang

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Image/video coding for machines. The most related works to our method are those on split computing [1,[36][37][38][39], where a pre-trained model is split into two parts and deployed on different devices, and the intermediate feature is compressed and then transmitted. However, to our knowledge, none of the existing works achieves end-to-end training of multiple splitting points as in ours.…”

Section: Related Workmentioning

confidence: 99%

Balancing the Encoder and Decoder Complexity in Image Compression for Classification

Duan,

Hossain,

et al. 2024

Preprint

View full text Add to dashboard Cite

This paper presents a study on the computational complexity of coding for machines, with a focus on image coding for classification.We first conduct a comprehensive set of experiments to analyze the size of the encoder (which encodes images to bitstreams), the size of the decoder (which decodes bitstreams and predicts class labels), and their impact on the rate-accuracy trade-off in compression for classification.Through empirical investigation, we demonstrate a complementary relationship between the encoder size and the decoder size, \ie, it is better to employ a large encoder with a small decoder and vice versa.Motivated by this relationship, we introduce a feature compression-based method for efficient image compression for classification.By compressing features at various layers of a neural network-based image classification model, our method achieves adjustable rate, accuracy, and encoder (or decoder) size using a single model.Experimental results on ImageNet classification show that our method achieves competitive results with existing methods while being much more flexible.The code will be made publicly available.

show abstract

“…In the experiment, we adopt uniform scalar to quantize the feature, which is one of the most widely-used quantizers in deep feature compression [1,5,2,6,8]. It is worth noting that our proposed bit allocation method can also work with nonlinear scalar quantizers.…”

Section: Comparisonmentioning

confidence: 99%

“…[7] proposed a lightweight compression method which performs clipping, entropy-constrained quantization and entropy coding to compress the feature tensors. [8] developed a feature channel arrangement method for the image/video-codec based feature compression framework. [9] explored transform (i.e., DCT) for more efficient intermediate deep feature compression.…”

Section: Introduction Compression For Intermediate Deep Learning Feat...mentioning

confidence: 99%

“…Quantization is one of the key techniques in deep feature compression, however, prior works did not pay enough attention to optimizing the quantizer against network output error and the corresponding rate-distortion optimization. Most works, including [1,5,2,6,8], only applied uniform or logarithmic quantizer to reduce the feature data volume. [7] proposed an entropy-constrained quantizer to optimize the compression ratio for the subsequent entropy coding module but not directly optimize for task performance.…”

Section: Introduction Compression For Intermediate Deep Learning Feat...mentioning

confidence: 99%

See 1 more Smart Citation

Channel-Wise Bit Allocation for Deep Visual Feature Quantization

Wang

Chen

Wang

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Intermediate deep visual feature compression and transmission is an emerging research topic, which enables a good balance among computing load, bandwidth usage and generalization ability for AI-based visual analysis in edge-cloud collaboration. Quantization and the corresponding ratedistortion optimization are the key techniques in deep feature compression. In this paper, by exploring the feature statistics and a greedy iterative algorithm, we propose a channel-wise bit allocation method for deep feature quantization optimizing for network output error. Given the limited rate and computational power, the proposed method can quantize features with small information loss. Moreover, the method also provides the option to handle the trade-offs between computational cost and quantization performance. Experimental results on ResNet and VGGNet features demonstrate the effectiveness of the proposed bit allocation method.

show abstract

Deep Feature Compression Using Spatio-Temporal Arrangement Toward Collaborative Intelligent World

Cited by 16 publications

References 32 publications

Deep feature extraction and compression for human–machine vision

Deep feature extraction and compression for human–machine vision

Balancing the Encoder and Decoder Complexity in Image Compression for Classification

Channel-Wise Bit Allocation for Deep Visual Feature Quantization

Contact Info

Product

Resources

About