Image Pre-Transformation for Recognition-Aware Image Compression

Suzuki, Satoshi; Takagi, Motohiro; Hayase, Kazuya; Onishi, Takayuki; Shimizu, Atsushi

doi:10.1109/icip.2019.8803275

Cited by 26 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…machine vision constraint, adopts the loss functions that target machine vision optimization to train end-to-end learned codecs. In [9,30,39,40,58], the task-driven losses are adopted. Namely, in these works, the downstream deep networks for machine vision are connected to the outputs of the codecs, and these two parts are trained jointly.…”

Section: Machine Vision Targeted Codecmentioning

confidence: 99%

“…In general, there are three paths that new VCM approaches in recent two years are developed along. The first branch stands on the basis of image/video coding and rebuilds the codecs towards machine vision, called Machine Vision Targeted Codec [30,58,74]. For these methods, they offer analytics-friendly images/videos, which can achieve better analytics performance with low bit-rates.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics

Yang¹,

Huang²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

As an emerging research practice leveraging recent advanced AI techniques, e.g. deep models based prediction and generation, Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression, and attempts to optimize compactness and efficiency jointly from a unified perspective of high accuracy machine vision and full fidelity human vision. With the rapid advances of deep feature representation and visual data compression in mind, in this paper, we summarize VCM methodology and philosophy based on existing academia and industrial efforts. The development of VCM follows a general rate-distortion optimization, and the categorization of key modules or techniques is established including features assisted coding, scalable coding, intermediate feature compression/optimization, and machine vision targeted codec, from broader perspectives of vision tasks, analytics resources, etc. From previous works, it is demonstrated that, although existing works attempt to reveal the nature of scalable representation in bits when dealing with machine and human vision tasks, there remains a rare study in the generality of low bit rate representation, and accordingly how to support a variety of visual analytic tasks. Therefore, we investigate a novel visual information compression for the analytics taxonomy problem to strengthen the capability of compact visual representations extracted from multiple tasks for visual analytics. A new perspective of task relationships versus compression is revisited. By keeping in mind the transferability among different machine vision tasks (e.g. high-level semantic and mid-level geometry-related), we aim to support multiple tasks jointly at low bit rates. In particular, to narrow the dimensionality gap between neural network generated features extracted from pixels and a variety of machine vision features/labels (e.g. scene class, segmentation labels), a codebook hyperprior is designed to compress the neural network-generated features. As demonstrated in our experiments, this new hyperprior model is expected to improve feature compression efficiency by estimating the signal entropy more accurately, which enables further investigation of the granularity of abstracting compact features among different tasks.

show abstract

Section: Machine Vision Targeted Codecmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics

Yang¹,

Huang²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The impact of compression on recognition has also been studied in [20,32,34]. Luo et al [20] show that JPEG quantization coefficients can be optimized to obtain lower bit-rates and at the same time preserve perceptual quality and recognition performance.…”

Section: Related Workmentioning

confidence: 99%

“…Luo et al [20] show that JPEG quantization coefficients can be optimized to obtain lower bit-rates and at the same time preserve perceptual quality and recognition performance. Recently, several preediting methods for more efficient compression without sacrifice of the classification accuracy have been introduced [32,35]. Typically these methods rely on some kind of rate-distortion-accuracy optimization.…”

Section: Related Workmentioning

confidence: 99%

Learning to Resize Images for Computer Vision Tasks

Talebi¹,

Milanfar²

2021

Preprint

View full text Add to dashboard Cite

For all the ways convolutional neural nets have revolutionized computer vision in recent years, one important aspect has received surprisingly little attention: the effect of image size on the accuracy of tasks being trained for. Typically, to be efficient, the input images are resized to a relatively small spatial resolution (e.g. 224 × 224), and both training and inference are carried out at this resolution. The actual mechanism for this re-scaling has been an afterthought: Namely, off-the-shelf image resizers such as bilinear and bicubic are commonly used in most machine learning software frameworks. But do these resizers limit the on-task performance of the trained networks? The answer is yes. Indeed, we show that the typical linear resizer can be replaced with learned resizers that can substantially improve performance. Importantly, while the classical resizers typically result in better perceptual quality of the downscaled images, our proposed learned resizers do not necessarily give better visual quality, but instead improve task performance.Our learned image resizer is jointly trained with a baseline vision model. This learned CNN-based resizer creates machine friendly visual manipulations that lead to a consistent improvement of the end task metric over the baseline model. Specifically, here we focus on the classification task with the ImageNet dataset [25], and experiment with four different models to learn resizers adapted to each model. Moreover, we show that the proposed resizer can also be useful for fine-tuning the classification baselines for other vision tasks. To this end, we experiment with three different baselines to develop image quality assessment (IQA) models on the AVA dataset [23].

show abstract

“…Second, our treatment is general, fusing the above and more modes of design into one holistic scheme. Indeed, our work could be considered as an extension of the broad view in [30,31,32] that proposed an optimization of a general image pre-processing stage while using a recognition-related or other losses.…”

Section: Related Workmentioning

confidence: 99%

The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study

Luo

Talebi

Yang

et al. 2020

Preprint

View full text Add to dashboard Cite

Default Optimized Default Optimized PSNR = 30.24dB PSNR = 30.62dB "Dragonfly" [0.16] "Bee Eater" [0.25] BPP = 3.06 BPP = 2.30 BPP = 1.706 BPP = 1.680 Fig. 1: We offer an optimization of JPEG's quantization tables for improved ratedistortion or rate-accuracy performance. The above are sample results showing (i) a 17% reduction in file size while maintaining the same quality, and (ii), a corrected classification while saving 1.5% in file size.

show abstract

Image Pre-Transformation for Recognition-Aware Image Compression

Cited by 26 publications

References 14 publications

Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics

Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics

Learning to Resize Images for Computer Vision Tasks

The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study

Contact Info

Product

Resources

About