Light-Weight CNN-Based VVC Inter Partitioning Acceleration

Abstract

The Versatile Video Coding (VVC) standard has been finalized by Joint Video Exploration Team (JVET) in 2020. Compared to the High Efficiency Video Coding (HEVC) standard, VVC offers about 50% compression efficiency gain, in terms of Bjontegaard Delta-Rate (BD-rate), at the cost of about 10x more encoder complexity [1]. In this paper, we propose a Convolutional Neural Network (CNN)-based method to speed up inter partitioning in VVC. Our method operates at the Coding Tree Unit (CTU) level, by splitting each CTU into a fixed grid of 8×8 blocks. Then each cell in this grid is associated with information about the partitioning depth within that area. A lightweight network for predicting this grid is employed during the rate-distortion optimization to limit the Quaternary Tree (QT)-split search and avoid partitions that are unlikely to be selected. Experiments show that the proposed method can achieve acceleration ranging from 17% to 30% in the Ran-domAccess Group Of Picture 32 (RAGOP32) mode of VVC Test Model (VTM)10 with a reasonable efficiency drop ranging from 0.37% to 1.18% in terms of BD-rate increase.

A classification‐prediction joint framework to accelerate QTMT‐based CU partition of inter‐mode VVC

“…Therefore, the partition of a 128× 128 CTU is represented as a 31× 31 PHM, whose each element is in {1, 2, 3, 4} reflecting the partition homogeneity of the collocated 8× 8 unit. Different from the previous structural maps in [6] and [7], the PHM incorporates the horizontal and vertical partition structures to model the QTMT‐based CTU partition as a value map, which is capable of driving our joint framework to achieve efficient partition prediction.…”

“…Experimental results: Table 1 shows the overall performance comparison of the proposed approach and state-of-the-art [5][6][7] methods. As shown in this table, the proposed approach achieves an average complexity reduction of 44.5% with 1.94% BD-BR increase.…”

“…To take full advantage of spatial and temporal information of the entire coding tree unit (CTU) and reduce the number of CNN models, end-to-end partition prediction methods represent the partition of the whole CTU as a structural map, and utilize a single CNN to predict it at once. For example, Liu et al [6] utilized CNN to predict the QT depth of each 8×8 block, and Tissier et al [7] adopted CNN to predict a structural map reflecting the probabilities of 4×4 split boundaries for CTU. While these end-to-end approaches exploit more spatial and temporal information, there still are two problems to be addressed:…”

“…For example, Liu et al. [6] utilized CNN to predict the QT depth of each 8× 8 block, and Tissier et al. [7] adopted CNN to predict a structural map reflecting the probabilities of 4× 4 split boundaries for CTU.…”

Portable Perceptron Network-Based Fast Mode Decision for Video-Based Point Cloud Compression

“…While large networks are effective, they hinder integration into practical applications. Lightweight neural network methods can also serve the purpose of accelerating encoding tasks [18][19][20][21][22][23] . For instance, Ryu and Kang [24] utilized a random forest method to accelerate angle prediction in intrapicture encoding, while Zhang et al [25] used SVM to predict the optimal depth of CU, both achieving significant results.…”