Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images

Xing, Qunliang; Xu, Mai; Li, Tianyi; Guan, Zhenyu

doi:10.1007/978-3-030-58517-4_17

Cited by 39 publications

(13 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Phuong et al [58] Vision/Classification Distillation-based EE from later to earlier MSDNet exits. RBQE [82] Vision/Enhancement UNet-like network with EE, for Quality Enhancement. The Right Tool for the Job [61] NLP Jointly trained EE on BERT.…”

Section: Open Challengesmentioning

confidence: 99%

Adaptive Inference through Early-Exit Networks

Laskaridis

Kouris

Lane

2021

Proceedings of the 5th International Workshop on Embedded and Mobile Deep Learning

View full text Add to dashboard Cite

DNNs are becoming less and less over-parametrised due to recent advances in efficient model design, through careful hand-crafted or NAS-based methods. Relying on the fact that not all inputs require the same amount of computation to yield a confident prediction, adaptive inference is gaining attention as a prominent approach for pushing the limits of efficient deployment. Particularly, early-exit networks comprise an emerging direction for tailoring the computation depth of each input sample at runtime, offering complementary performance gains to other efficiency optimisations. In this paper, we decompose the design methodology of early-exit networks to its key components and survey the recent advances in each one of them. We also position early-exiting against other efficient inference solutions and provide our insights on the current challenges and most promising future directions for research in the field. CCS CONCEPTS• Computing methodologies → Neural networks; • Humancentered computing → Ubiquitous and mobile computing systems and tools.

show abstract

Section: Open Challengesmentioning

confidence: 99%

Adaptive Inference through Early-Exit Networks

Laskaridis

Kouris

Lane

2021

Proceedings of the 5th International Workshop on Embedded and Mobile Deep Learning

View full text Add to dashboard Cite

show abstract

“…Furthermore, in some works, the focus is put on strategies for enhancing the quality of video frames. In [53], a blind quality enhancement approach is proposed where frames with different distortion levels are processed differently. An "easyto-hard" QE strategy is used to determine which level of the CNN-based filtering shall be applied on a given frame.…”

Section: A Single-frame Quality Enhancementmentioning

confidence: 99%

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Nasiri

Hamidouche

Morin

et al. 2021

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

show abstract

“…During the past a few years, plenty of works apply DNNs to improve the performance of image compression. Among them, [12,13,36,40,28,20,10,39,38,37] proposed post-processing DNNs to enhance the compressed image without bitrate overhead, thus improving the rate-distortion performance. For example, Dong et al [12] proposed a four-layer DNN to reduce the compression artifacts of JPEG.…”

Section: Related Workmentioning

confidence: 99%

“…Meanwhile, some approaches aim at the post-processing of the images compressed by the HEVC [27] intra mode, e.g., Dai et al [10], DS-CNN-I [39] and QE-CNN-I [38]. Most recently, Xing et al [37] proposed a dynamic DNN to blindly enhance the images compressed with different quality, and showed the effectiveness on both JPEG-and HEVC-compressed images.…”

Section: Related Workmentioning

confidence: 99%

Learning to Improve Image Compression without Changing the Standard Decoder

Strümpler¹,

Ren²,

Timofte³

2020

Preprint

View full text Add to dashboard Cite

In recent years we have witnessed an increasing interest in applying Deep Neural Networks (DNNs) to improve the rate-distortion performance in image compression. However, the existing approaches either train a post-processing DNN on the decoder side, or propose learning for image compression in an end-to-end manner. This way, the trained DNNs are required in the decoder, leading to the incompatibility to the standard image decoders (e.g., JPEG) in personal computers and mobiles. Therefore, we propose learning to improve the encoding performance with the standard decoder. In this paper, We work on JPEG as an example. Specifically, a frequency-domain pre-editing method is proposed to optimize the distribution of DCT coefficients, aiming at facilitating the JPEG compression. Moreover, we propose learning the JPEG quantization table jointly with the pre-editing network. Most importantly, we do not modify the JPEG decoder and therefore our approach is applicable when viewing images with the widely used standard JPEG decoder. The experiments validate that our approach successfully improves the rate-distortion performance of JPEG in terms of various quality metrics, such as PSNR, MS-SSIM and LPIPS. Visually, this translates to better overall color retention especially when strong compression is applied. The codes are available at https://github.com/ YannickStruempler/LearnedJPEG.

show abstract

Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images

Cited by 39 publications

References 41 publications

Adaptive Inference through Early-Exit Networks

Adaptive Inference through Early-Exit Networks

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Learning to Improve Image Compression without Changing the Standard Decoder

Contact Info

Product

Resources

About