The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study

Luo, Xiaoqiang; Talebi, Hossein; Yang, Feng; Elad, Michael; Milanfar, Peyman

doi:10.48550/arxiv.2008.00605

Cited by 8 publications

(13 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Related Workmentioning

confidence: 99%

“…The impact of compression on recognition has also been studied in [20,32,34]. Luo et al [20] show that JPEG quantization coefficients can be optimized to obtain lower bit-rates and at the same time preserve perceptual quality and recognition performance. Recently, several preediting methods for more efficient compression without sacrifice of the classification accuracy have been introduced [32,35].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Learning to Resize Images for Computer Vision Tasks

Talebi¹,

Milanfar²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

For all the ways convolutional neural nets have revolutionized computer vision in recent years, one important aspect has received surprisingly little attention: the effect of image size on the accuracy of tasks being trained for. Typically, to be efficient, the input images are resized to a relatively small spatial resolution (e.g. 224 × 224), and both training and inference are carried out at this resolution. The actual mechanism for this re-scaling has been an afterthought: Namely, off-the-shelf image resizers such as bilinear and bicubic are commonly used in most machine learning software frameworks. But do these resizers limit the on-task performance of the trained networks? The answer is yes. Indeed, we show that the typical linear resizer can be replaced with learned resizers that can substantially improve performance. Importantly, while the classical resizers typically result in better perceptual quality of the downscaled images, our proposed learned resizers do not necessarily give better visual quality, but instead improve task performance.Our learned image resizer is jointly trained with a baseline vision model. This learned CNN-based resizer creates machine friendly visual manipulations that lead to a consistent improvement of the end task metric over the baseline model. Specifically, here we focus on the classification task with the ImageNet dataset [25], and experiment with four different models to learn resizers adapted to each model. Moreover, we show that the proposed resizer can also be useful for fine-tuning the classification baselines for other vision tasks. To this end, we experiment with three different baselines to develop image quality assessment (IQA) models on the AVA dataset [23].

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Learning to Resize Images for Computer Vision Tasks

Talebi¹,

Milanfar²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Doing so, they achieve better compression efficiency without detriment in quality. This idea is expanded by [12,15] who use a larger Q-value search space and rely on further hyper-parameter tuning. Similar joint learning strategies are exploited in QuanNet [4] to optimise the quantization intervals of the JPEG2000 encoder, and by [3] to tune the weights of JPEG XS.…”

Section: Computer Vision Tasks Vs Jpeg Compressionmentioning

confidence: 99%

TACTIC: Joint Rate-Distortion-Accuracy Optimisation for Low Bitrate Compression

Kubiak,

Hadfield

2021

Preprint

View full text Add to dashboard Cite

We present TACTIC: Task-Aware Compression Through Intelligent Coding. Our lossy compression model learns based on the rate-distortion-accuracy trade-off for a specific task. By considering what information is important for the follow-on problem, the system trades off visual fidelity for good task performance at a low bitrate. When compared against JPEG at the same bitrate, our approach is able to improve the accuracy of ImageNet subset classification by 4.5%. We also demonstrate the applicability of our approach to other problems, providing a 3.4% accuracy and 4.9% mean IoU improvements in performance over task-agnostic compression for semantic segmentation.

show abstract

“…In terms of optimizing image/video compression codecs for a given learning task, only a handful of works emerges from the literature. The authors of [11] and [12] used an approximate differentiable codec implementation to learn better quantization tables for JPEG and JPEG2000. The method presented in [11] shows performance gain in rate-distortion-accuracy for ImageNet-1K by training the differentiable codec with a fixed MobileNet architecture [13].…”

Section: Introductionmentioning

confidence: 99%

“…The authors of [11] and [12] used an approximate differentiable codec implementation to learn better quantization tables for JPEG and JPEG2000. The method presented in [11] shows performance gain in rate-distortion-accuracy for ImageNet-1K by training the differentiable codec with a fixed MobileNet architecture [13]. However, the rate-accuracy performance in [11,12] is still limited by coding tools designed for visual quality.…”

Section: Introductionmentioning

confidence: 99%

End-to-end optimized image compression for machines, a study

Chamain¹,

Racapé²,

Bégaint³

et al. 2020

Preprint

View full text Add to dashboard Cite

An increasing share of image and video content is analyzed by machines rather than viewed by humans, and therefore it becomes relevant to optimize codecs for such applications where the analysis is performed remotely. Unfortunately, conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. However, neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model. In this paper, we propose to study an end-to-end framework enabling efficient image compression for remote machine task analysis, using a chain composed of a compression module and a task algorithm that can be optimized end-to-end. We show that it is possible to significantly improve the task accuracy when fine-tuning jointly the codec and the task networks, especially at low bitrates. Depending on training or deployment constraints, selective fine-tuning can be applied only on the encoder, decoder or task network and still achieve rate-accuracy improvements over an off-the-shelf codec and task network. Our results also demonstrate the flexibility of end-to-end pipelines for practical applications. ‡ Equal contribution.This work was done while Lahiru D. Chamain was an intern at the InterDigital AI Lab.

show abstract

The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study

Cited by 8 publications

References 36 publications

Learning to Resize Images for Computer Vision Tasks

Learning to Resize Images for Computer Vision Tasks

TACTIC: Joint Rate-Distortion-Accuracy Optimisation for Low Bitrate Compression

End-to-end optimized image compression for machines, a study

Contact Info

Product

Resources

About