A CNN-Based Fast Inter Coding Method for VVC

Pan, Zhaoqing; Zhang, Peihan; Peng, Bo; Ling, Nam; Lei, Jianjun

doi:10.1109/lsp.2021.3086692

Cited by 50 publications

(52 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ 35 ] proposed a CNN-based fast-partition method. Similar studies have been conducted in [ 36 , 37 ]. By jointly using multi-domain information, Pan et al.…”

Section: Related Worksupporting

confidence: 71%

See 1 more Smart Citation

Low-Complexity Multiple Transform Selection Combining Multi-Type Tree Partition Algorithm for Versatile Video Coding

Xiong

Yang

et al. 2022

Sensors

View full text Add to dashboard Cite

Despite the fact that Versatile Video Coding (VVC) achieves a superior coding performance to High-Efficiency Video Coding (HEVC), it takes a lot of time to encode video sequences due to the high computational complexity of the tools. Among these tools, Multiple Transform Selection (MTS) require the best of several transforms to be obtained using the Rate-Distortion Optimization (RDO) process, which increases the time spent video encoding, meaning that VVC is not suited to real-time sensor application networks. In this paper, a low-complexity multiple transform selection, combined with the multi-type tree partition algorithm, is proposed to address the above issue. First, to skip the MTS process, we introduce a method to estimate the Rate-Distortion (RD) cost of the last Coding Unit (CU) based on the relationship between the RD costs of transform candidates and the correlation between Sub-Coding Units’ (sub-CUs’) information entropy under binary splitting. When the sum of the RD costs of sub-CUs is greater than or equal to their parent CU, the RD checking of MTS will be skipped. Second, we make full use of the coding information of neighboring CUs to terminate MTS early. The experimental results show that, compared with the VVC, the proposed method achieves a 26.40% reduction in time, with a 0.13% increase in Bjøontegaard Delta Bitrate (BDBR).

show abstract

“…[ 35 ] proposed a CNN-based fast-partition method. Similar studies have been conducted in [ 36 , 37 ]. By jointly using multi-domain information, Pan et al.…”

Section: Related Worksupporting

confidence: 71%

“…By jointly using multi-domain information, Pan et al. [ 37 ] introduced a fast inter-coding method to terminate the CU partition process early. A Hierarchy Grid Fully Convolutional Network (HG-FCN) framework was proposed in [ 38 ] to effectively predict the quad-tree with a nested multi-type tree (QTMT).…”

Section: Related Workmentioning

confidence: 99%

Low-Complexity Multiple Transform Selection Combining Multi-Type Tree Partition Algorithm for Versatile Video Coding

Xiong

Yang

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The proposed method offers much better tradeoff between time saving and performance drop than the state of the art methods. More precisely, when we set the threshold Th in Algorithm 1 to 0, we obtain a similar acceleration as the method developed in [8] with a significantly lower loss. Similarly, when the threshold Th equals 0.1, acceleration is tripled and the loss remains the same as in [9].…”

Section: Resultsmentioning

confidence: 91%

“…In Table 1, the proposed method is compared with the two CNN-based inter partitioning methods in [8] and [9]. For a fair comparison, we have implemented our method in VTM6 and VTM11.…”

Section: Resultsmentioning

confidence: 99%

“…This method has been later improved by Kulupana et al [7], by combining it with a handcrafted early termination rule for the Ternary-type Tree (TT) splits. For CNN-based methods, Pan et al propose in [8] a multi-branch CNN, namely MF-CNN, to perform a binary classification of the "Partition" or "Non-partition" at the Coding Unit (CU) level, in order to early terminate the partitioning. In this paper, different models are trained for different CU sizes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Light-Weight CNN-Based VVC Inter Partitioning Acceleration

Liu¹,

Abdoli²,

Guionnet³

et al. 2022

2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP)

View full text Add to dashboard Cite

The Versatile Video Coding (VVC) standard has been finalized by Joint Video Exploration Team (JVET) in 2020. Compared to the High Efficiency Video Coding (HEVC) standard, VVC offers about 50% compression efficiency gain, in terms of Bjontegaard Delta-Rate (BD-rate), at the cost of about 10x more encoder complexity [1]. In this paper, we propose a Convolutional Neural Network (CNN)-based method to speed up inter partitioning in VVC. Our method operates at the Coding Tree Unit (CTU) level, by splitting each CTU into a fixed grid of 8×8 blocks. Then each cell in this grid is associated with information about the partitioning depth within that area. A lightweight network for predicting this grid is employed during the rate-distortion optimization to limit the Quaternary Tree (QT)-split search and avoid partitions that are unlikely to be selected. Experiments show that the proposed method can achieve acceleration ranging from 17% to 30% in the Ran-domAccess Group Of Picture 32 (RAGOP32) mode of VVC Test Model (VTM)10 with a reasonable efficiency drop ranging from 0.37% to 1.18% in terms of BD-rate increase.

show abstract