Enhancing VVC Through Cnn-Based Post-Processing

Zhang, Fan; Feng, Chen; Bull, David

doi:10.1109/icme46284.2020.9102912

Cited by 40 publications

(16 citation statements)

References 23 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Postprocessing is commonly applied at the video decoder, on the reconstructed frames, to reduce various coding artefacts and enhance visual quality. Here, we employed the CNN-based postprocessing method proposed by Zhang et al [16], which has been reported to offer significant coding gains over VVC. Its network architecture was modified based on the generator (SRResNet) of SRGAN [50].…”

Section: Postprocessing Methods With Image Enhancementmentioning

confidence: 99%

See 1 more Smart Citation

Encoding in the Dark Grand Challenge: an Overview

Anantrasirichai

Zhang

Malyugina

et al. 2020

2020 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

Self Cite

View full text Add to dashboard Cite

A big part of the video content we consume from video providers consists of genres featuring low-light aesthetics. Low light sequences have special characteristics, such as spatio-temporal varying acquisition noise and light flickering, that make the encoding process challenging. To deal with the spatio-temporal incoherent noise, higher bitrates are used to achieve high objective quality. Additionally, the quality assessment metrics and methods have not been designed, trained or tested for this type of content. This has inspired us to trigger research in that area and propose a Grand Challenge on encoding low-light video sequences. In this paper, we present an overview of the proposed challenge, and test state-of-the-art methods that will be part of the benchmark methods at the stage of the participants' deliverable assessment. From this exploration, our results show that VVC already achieves a high performance compared to simply denoising the video source prior to encoding. Moreover, the quality of the video streams can be further improved by employing a post-processing image enhancement method.

show abstract

Section: Postprocessing Methods With Image Enhancementmentioning

confidence: 99%

“…In heavy compression, quantization filters out the noise [12]. CNN-based methods are also popular for postprocessing and can provide significant image enhancement, leading to better final ratedistortion performance with significantly lower complexity [15,16].…”

Section: Introductionmentioning

confidence: 99%

Encoding in the Dark Grand Challenge: an Overview

Anantrasirichai

Zhang

Malyugina

et al. 2020

2020 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Filtering in the decoder side [32], [54], [55] can also solve the problem of repeated enhancement well. For example, DS-CNN was designed by Yao et al [32] to achieve quality enhancement as well.…”

Section: Related Workmentioning

confidence: 99%

“…Li et al [54] adopted a 20-layers deep CNN to improve the filtering performance. Zhang et al [55] proposed a post-processing network for VTM 4.0.1. In summary, filtering in inter frames is more challenging than that of intra frames.…”

Section: Related Workmentioning

confidence: 99%

A Convolutional Neural Network-Based Low Complexity Filter

Liu,

Sun,

Katto

et al. 2020

Preprint

View full text Add to dashboard Cite

Convolutional Neural Network (CNN)-based filters have achieved significant performance in video artifacts reduction. However, the high complexity of existing methods makes it difficult to be applied in real usage. In this paper, a CNN-based low complexity filter is proposed. We utilize depth separable convolution (DSC) merged with the batch normalization (BN) as the backbone of our proposed CNN-based network. Besides, a weight initialization method is proposed to enhance the training performance. To solve the well known over smoothing problem for the inter frames, a frame-level residual mapping (RM) is presented. We analyze some of the mainstream methods like frame-level and block-level based filters quantitatively and build our CNN-based filter with frame-level control to avoid the extra complexity and artificial boundaries caused by block-level control. In addition, a novel module called RM is designed to restore the distortion from the learned residuals. As a result, we can effectively improve the generalization ability of the learningbased filter and reach an adaptive filtering effect. Moreover, this module is flexible and can be combined with other learningbased filters. The experimental results show that our proposed method achieves significant BD-rate reduction than H.265/HEVC. It achieves about 1.2% BD-rate reduction and 79.1% decrease in FLOPs than VR-CNN. Finally, the measurement on H.266/VVC and ablation studies are also conducted to ensure the effectiveness of the proposed method.

show abstract

“…This paper is a comprehensive extension of our previous work [3], which solely focused on the PSNR driven optimization of VVC compressed content. The primary differences are summarized below:…”

Section: Introductionmentioning

confidence: 99%

Video Compression with CNN-based Post Processing

Zhang,

Ma,

Feng

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, video compression techniques have been significantly challenged by the rapidly increased demands associated with high quality and immersive video content. Among various compression tools, post-processing can be applied on reconstructed video content to mitigate visible compression artefacts and to enhance overall perceptual quality. Inspired by advances in deep learning, we propose a new CNN-based post-processing approach, which has been integrated with two state-of-the-art coding standards, VVC and AV1. The results show consistent coding gains on all tested sequences at various spatial resolutions, with average bit rate savings of 4.0% and 5.8% against original VVC and AV1 respectively (based on the assessment of PSNR). This network has also been trained with perceptually inspired loss functions, which have further improved reconstruction quality based on perceptual quality assessment (VMAF), with average coding gains of 13.9% over VVC and 10.5% against AV1.

show abstract

Enhancing VVC Through Cnn-Based Post-Processing

Cited by 40 publications

References 23 publications

Encoding in the Dark Grand Challenge: an Overview

Encoding in the Dark Grand Challenge: an Overview

A Convolutional Neural Network-Based Low Complexity Filter

Video Compression with CNN-based Post Processing

Contact Info

Product

Resources

About