Biprediction-Based Video Quality Enhancement via Learning

Ding, Dandan; Wang, Wenyu; Tong, Junchao; Gao, Xinbo; Liu, Zoe; Fang, Yong

doi:10.1109/tcyb.2020.2998481

“…• QP-specific training: dedicating one model for each QP or a range of QPs [8,66]. • QP-map training: providing QP as an input to the network [12,27,33,50,57]. Each approach has benefits and drawbacks.…”

Section: Methods Based On Coding Informationmentioning

confidence: 99%

“…Inspired by the multi-frame QE methods, a bi-prediction approach is proposed in [57]. In this work, instead of computing the motion field, a prediction of the current low quality frame is generated from neighbouring high quality frames.…”

Section: B Multi-frame Quality Enhancementmentioning

confidence: 99%

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Nasiri

¹

,

Hamidouche

²

,

Morin

³

et al. 2021

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

show abstract

“…• QP-specific training: dedicating one model for each QP or a range of QPs [8,66]. • QP-map training: providing QP as an input to the network [12,27,33,50,57]. Each approach has benefits and drawbacks.…”

Section: Methods Based On Coding Informationmentioning

confidence: 99%

Prediction-Aware Quality Enhancement of VVC Using CNN

Nasiri

¹

,

Hamidouche

²

,

Morin

³

et al. 2020

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

View full text Add to dashboard Cite

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

show abstract

“…Ding et al propose a two-stage network to implement the synthesisation and fusion respectively. It leverages the cross-correlation between successive frames for enhancement thus achieves high accuracy performance [35].…”

Section: Video Quality Enhancementmentioning

confidence: 99%

An efficient Industrial Internet of Things video data processing system for protocol identification and quality enhancement

Chen

¹

,

Liu

²

,

Zhang

³

2022

IET Cyber-Phy Sys Theory & Ap

1

0

View full text Add to dashboard Cite

Video has become an essential medium to monitoring, identification and knowledge sharing. For industrial applications, especially Industrial Internet of Things (IIoT), videos encoded with specific protocols are transferred to smart gateways. In a typical IIoT scenario, the protocol of the video is firstly recognised, which prepares for subsequent video tasks. Due to the constrained resources in such scenarios, the video quality can be deteriorated during encoding and compression processes, which is challenging for IIoT. Recently, there have been extensive works focussing on the protocol identification (PI) and video quality enhancement (VQE) tasks on IIoT edge devices using deep neural networks (DNNs). Since DNNs often require high computational resources, complex networks can hardly be deployed on edge devices. An IIoT system which can efficiently identify the stream protocol and enhance the video quality is proposed in this study. The light‐weighted network designs and inference optimisation techniques have been proposed for PI and VQE to realise efficient deployments. Our proposed system employed on an IIoT edge device can achieve an accuracy of higher than 97.52% with fast inference speed for PI. For the VQE task, our system has demonstrated superior performance (15.230 FPS, 0.773 FPS/W) in comparison with the state‐of‐the‐art methods.

show abstract

Biprediction-Based Video Quality Enhancement via Learning

Cited by 12 publications

References 47 publications

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Prediction-Aware Quality Enhancement of VVC Using CNN

An efficient Industrial Internet of Things video data processing system for protocol identification and quality enhancement

Contact Info

Product

Resources

About