A Switchable Deep Learning Approach for In-Loop Filtering in Video Coding

Ding, Dandan; Kong, Lingyi; Chen, Guangyao; Liu, Zoe; Fang, Yong

doi:10.1109/tcsvt.2019.2935508

Cited by 63 publications

(35 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subsequently, I1 is the reference of I2 and so forth. 3 If frame I1 is filtered again by the same CNN model, an overfiltering problem will be triggered, resulting in severely degraded performance, as analyzed in [170]. To overcome this challenging problem, a CNN model called SimNet was built to carry the relationship between the reconstructed frame and its original frame in [171] to adaptively skip filtering operations in intercoding.…”

Section: A In-loop Filteringmentioning

confidence: 99%

Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

Ding

Chen

et al. 2021

Proc. IEEE

Self Cite

View full text Add to dashboard Cite

Significant advances in video compression systems have been made in the past several decades to satisfy the near-exponential growth of Internet-scale video traffic. From the application perspective, we have identified three major functional blocks, including preprocessing, coding, and postprocessing, which have been continuously investigated to maximize the end-user quality of experience (QoE) under a limited bit rate budget. Recently, artificial intelligence (AI)-powered techniques have shown great potential to further increase the efficiency of the aforementioned functional blocks, both individually and jointly. In this article, we review recent technical advances in video compression systems extensively, with an emphasis on deep neural network (DNN)based approaches, and then present three comprehensive case studies. On preprocessing, we show a switchable texturebased video coding example that leverages DNN-based scene understanding to extract semantic areas for the improvement Manuscript

show abstract

Section: A In-loop Filteringmentioning

confidence: 99%

Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

Ding

Chen

et al. 2021

Proc. IEEE

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the case that the quality is already satisfying based on a blind quality assessment metric, the QE process stops, otherwise, it continues. In Squeeze-and-Excitation Filtering CNN (SEFCNN) [43], an adaptive ILF is also proposed in which networks with various complexity levels are trained for different QPs.…”

Section: A Single-frame Quality Enhancementmentioning

confidence: 99%

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Nasiri

Hamidouche

Morin

et al. 2021

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

show abstract

“…In [18], diamond adaptive rod pattern-search-algorithm-based block-matching motion estimation algorithms were proposed for multistranded codec hardware design to provide a high compression rate with less computational complexity. In [19], the authors proposed a deep-learning-based systematic approach that included an effective convolutional neural network structure, hierarchical training strategy, and video-codec-oriented switchable mechanism.…”

Section: Related Workmentioning

confidence: 99%

A Novel Model for Large-Scale Online College Learning in Postpandemic Era: AI-Driven Approach

Wang

2021

Mobile Information Systems

View full text Add to dashboard Cite

COVID-19 is a pandemic with a wide reach and explosive magnitude, and the world has been bracing itself for impact. Many have lost their jobs and savings, and many are homeless. For better or worse, COVID-19 has permanently changed our lives. For college students, the pandemic means giving up most of the on-campus experience in the postpandemic era and performing online learning instead. Virtual lessons may become a permanent part of college education. Large-scale online learning typically utilizes interactive live video streaming. In this study, we analyzed a codec and video streaming transmission protocol using artificial intelligence. First, we studied an intraframe prediction optimization algorithm for the H.266 codec based on long short-term memory networks. In terms of video streaming transmission protocols, real-time communication optimization based on Quick UDP Internet connections and Luby Transform codes is proposed to improve the quality of interactive live video streaming. Experimental results demonstrate that the proposed strategy outperforms three benchmarks in terms of video streaming quality, video streaming latency, and average throughput.

show abstract

A Switchable Deep Learning Approach for In-Loop Filtering in Video Coding

Cited by 63 publications

References 33 publications

Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

Advances in Video Compression System Using Deep Neural Network: A Review and Case Studies

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

A Novel Model for Large-Scale Online College Learning in Postpandemic Era: AI-Driven Approach

Contact Info

Product

Resources

About