Deep Affine Motion Compensation Network for Inter Prediction in VVC

Jin, Dengchao; Lei, Jianjun; Peng, Bo; Li, Wanqing; Ling, Nam; Huang, Qingming

doi:10.1109/tcsvt.2021.3107135

Cited by 18 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It further advances the rate-distortion performance with fewer parameters than the previous works [61] and [62]. Most recently, Jin et al [64] proposed a deep affine motion compensation network to deal with the deformable motion compensation and applied the proposed method in VVC [13].…”

Section: Related Workmentioning

confidence: 99%

Advancing Learned Video Compression with In-loop Frame Prediction

Yang,

Timofte,

Van Gool

2022

Preprint

View full text Add to dashboard Cite

Recent years have witnessed an increasing interest in end-to-end learned video compression. Most previous works explore temporal redundancy by detecting and compressing a motion map to warp the reference frame towards the target frame. Yet, it failed to adequately take advantage of the historical priors in the sequential reference frames. In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module, which is able to effectively predict the target frame from the previously compressed frames, without consuming any bit-rate. The predicted frame can serve as a better reference than the previously compressed frame, and therefore it benefits the compression performance. The proposed in-loop prediction module is a part of the end-to-end video compression and is jointly optimized in the whole framework. We propose the recurrent and the bi-directional in-loop prediction modules for compressing Pframes and B-frames, respectively. The experiments show the state-of-the-art performance of our ALVC approach in learned video compression. We also outperform the default hierarchical B mode of x265 in terms of PSNR and beat the slowest mode of the SSIM-tuned x265 on MS-SSIM. The project page: https://github.com/RenYang-home/ALVC.

show abstract

Section: Related Workmentioning

confidence: 99%

Advancing Learned Video Compression with In-loop Frame Prediction

Yang,

Timofte,

Van Gool

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The range of the intra-frame angle prediction mode has been increased from 35 to 67. In the same time, VTM added many new methods, such as inter-frame local illumination compensation (LIC), bi-directional optical flow prediction (BIO), affine motion compensation (AMC) [3], and adaptive intra-loop filter (ALF) [4], which significantly improve the compression performance, but also bring huge computational complexity. In the All-Intra test configuration, the VVC codec can reduce bit rate by 30% to 50% while maintaining perceptual quality that is comparable to HEVC, but this benefit comes at the expense of coding complexity that is 10 times higher than that of HEVC.…”

Section: Introductionmentioning

confidence: 99%

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

2023

View full text Add to dashboard Cite

In order to satisfy the growing need for high-quality video, VVC comes with more efficient coding performance. According to statistical analysis, the level of coding complexity in VVC is tenfold greater compared to that of HEVC, so it is our main goal to study that what methods can be employed to decrease the time complexity of VVC. CU split in intra-frame modes requires the split mode decision by RD loss calculation, and the process of coding makes it to calculate RD calculation for all possible mode combinations, which is an important area that brings complexity to video coding, so in order to achieve our goal. Initially, we introduce an optimal depth prediction algorithm for Coding Units (CUs) by leveraging temporal combination. This algorithm collects depth information of CUs to predict the coding depth of CU blocks. Additionally, we suggest a decision tree-based method for CU split mode decision. With this method, we can make a decision on the CU split mode within the obtained split depth, reducing the time complexity of coding. This decision is based on the predictions from the first algorithm. The results demonstrate that our algorithm achieves superior performance over state-of-the-art methods in terms of computational complexity and compression quality. Compared to the VVC reference software (VTM), our method saves an average of 53.92% in coding time and improves the BDBR by 1.74%. These findings suggest that our method is highly effective in improving both computational efficiency and compression quality.

show abstract

“…(1) The traditional statistical method has limited ability to reduce the coding complexity, and it does not strike a good balance between coding efficiency and coding computational complexity [19]. (2) Machine learning methods need to extract image features [20], which brings additional overhead. In this work, an efficient extreme learning machine-based CU size decision method is proposed to reduce the coding complexity for VVC inter-prediction.…”

Section: Introductionmentioning

confidence: 99%

Extreme Learning Machine-Enabled Coding Unit Partitioning Algorithm for Versatile Video Coding

Jiang,

Xiang,

Jin

et al. 2023

Information

View full text Add to dashboard Cite

The versatile video coding (VVC) standard offers improved coding efficiency compared to the high efficiency video coding (HEVC) standard in multimedia signal coding. However, this increased efficiency comes at the cost of increased coding complexity. This work proposes an efficient coding unit partitioning algorithm based on an extreme learning machine (ELM), which can reduce the coding complexity while ensuring coding efficiency. Firstly, the coding unit size decision is modeled as a classification problem. Secondly, an ELM classifier is trained to predict the coding unit size. In the experiment, the proposed approach is verified based on the VVC reference model. The results show that the proposed method can reduce coding complexity significantly, and good image quality can be obtained.

show abstract

Deep Affine Motion Compensation Network for Inter Prediction in VVC

Cited by 18 publications

References 38 publications

Advancing Learned Video Compression with In-loop Frame Prediction

Advancing Learned Video Compression with In-loop Frame Prediction

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

Extreme Learning Machine-Enabled Coding Unit Partitioning Algorithm for Versatile Video Coding

Contact Info

Product

Resources

About