Multitask Learning for VVC Quality Enhancement and Super-Resolution

Bonnineau, Charles; Hamidouche, Wassim; Travers, Jean-Francois; Sidaty, Naty; Déforges, Olivier

doi:10.1109/pcs50896.2021.9477492

Cited by 8 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed method in [ 27 ] showed a notable enhancement in perceptual visual quality, achieving a reduction of 3.9% in performance on the BD-BR for the RA configuration compared to VVC/H.266. Bonnineau et al [ 28 ] introduced a multitask learning-based approach that employed a QP map to generalize the model with various QPs by sharing parameters within a single network and task-specific modules. The method presented in [ 28 ] exhibited a significant improvement in perceptual visual quality, achieving a reduction of 2.8% in performance on the BD-BR for the RA configuration compared to VVC/H.266.…”

Section: Related Workmentioning

confidence: 99%

“…Bonnineau et al [ 28 ] introduced a multitask learning-based approach that employed a QP map to generalize the model with various QPs by sharing parameters within a single network and task-specific modules. The method presented in [ 28 ] exhibited a significant improvement in perceptual visual quality, achieving a reduction of 2.8% in performance on the BD-BR for the RA configuration compared to VVC/H.266. Wang et al [ 29 ] aimed to enhance the visual quality of decoded images by incorporating partitioning information with QP information, introducing a three-branch network.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Block Partitioning Information-Based CNN Post-Filtering for EVC Baseline Profile

Choi

2024

Sensors

View full text Add to dashboard Cite

The need for efficient video coding technology is more important than ever in the current scenario where video applications are increasing worldwide, and Internet of Things (IoT) devices are becoming widespread. In this context, it is necessary to carefully review the recently completed MPEG-5 Essential Video Coding (EVC) standard because the EVC Baseline profile is customized to meet the specific requirements needed to process IoT video data in terms of low complexity. Nevertheless, the EVC Baseline profile has a notable disadvantage. Since it is a codec composed only of simple tools developed over 20 years, it tends to represent numerous coding artifacts. In particular, the presence of blocking artifacts at the block boundary is regarded as a critical issue that must be addressed. To address this, this paper proposes a post-filter using a block partitioning information-based Convolutional Neural Network (CNN). The proposed method in the experimental results objectively shows an approximately 0.57 dB for All-Intra (AI) and 0.37 dB for Low-Delay (LD) improvements in each configuration by the proposed method when compared to the pre-post-filter video, and the enhanced PSNR results in an overall bitrate reduction of 11.62% for AI and 10.91% for LD in the Luma and Chroma components, respectively. Due to the huge improvement in the PSNR, the proposed method significantly improved the visual quality subjectively, particularly in blocking artifacts at the coding block boundary.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Block Partitioning Information-Based CNN Post-Filtering for EVC Baseline Profile

Choi

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…At low-bitrate, this process may provide better coding performance than full-resolution coding [3] while enabling spatial scalability with any base-layer codec. With the recent advances in deep learning, powerful pre and post-processing models have been used for spatial resolution adaptation based on existing compression standards [4]- [7]. However, some high frequencies lost during the downscaling process still cannot be recovered using single post-processing modules, making performance sensitive to the content.…”

Section: Introductionmentioning

confidence: 99%

CAESR: Conditional Autoencoder and Super-Resolution for Learned Spatial Scalability

Bonnincau

Hamidouche

Travers

et al. 2021

2021 International Conference on Visual Communications and Image Processing (VCIP)

Self Cite

View full text Add to dashboard Cite

In this paper, we present CAESR, an hybrid learning-based coding approach for spatial scalability based on the versatile video coding (VVC) standard. Our framework considers a low-resolution signal encoded with VVC intra-mode as a base-layer (BL), and a deep conditional autoencoder with hyperprior (AE-HP) as an enhancement-layer (EL) model. The EL encoder takes as inputs both the upscaled BL reconstruction and the original image. Our approach relies on conditional coding that learns the optimal mixture of the source and the upscaled BL image, enabling better performance than residual coding. On the decoder side, a super-resolution (SR) module is used to recover high-resolution details and invert the conditional coding process. Experimental results have shown that our solution is competitive with the VVC full-resolution intra coding while being scalable.

show abstract

High Quality Video Frames From VVC: A Deep Neural Network Approach

Das

Choi

2023

IEEE Access

View full text Add to dashboard Cite

In recent years, video content has become a significant contributor to Internet traffic, prompting the development of efficient codecs, such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC), to reduce bandwidth usage and storage requirements. However, these video coding standards still exhibit quality degradation and artifacts in the decoded frames. To address this issue, researchers have introduced several network architectures based on deep-learning algorithms; however, most of them focus on in-loop filtering, which requires additional bits to transmit filter information from the encoder to the decoder under a video-coding framework. In this paper, we propose a neural-network-based post-processing method to enhance the decoded frames. In the experimental result, the proposed model achieves a significant bitrate reduction, as measured by Bjøntegaard Delta of 4.54%, 4.13%, and 5.21% for random access (RA), low-delay (LD), and all-intra (AI) configurations, respectively, while also improving peak signal-to-noise ratio (PSNR). VVC, post-processing, video compression, CNN. INDEX TERMS

show abstract

Multitask Learning for VVC Quality Enhancement and Super-Resolution

Cited by 8 publications

References 12 publications

Block Partitioning Information-Based CNN Post-Filtering for EVC Baseline Profile

Block Partitioning Information-Based CNN Post-Filtering for EVC Baseline Profile

CAESR: Conditional Autoencoder and Super-Resolution for Learned Spatial Scalability

High Quality Video Frames From VVC: A Deep Neural Network Approach

Contact Info

Product

Resources

About