Efficient VVC Intra Prediction Based on Deep Feature Fusion and Probability Estimation

Zhao, Tiesong; Huang, Yuhang; Feng, Weize; Xu, Yang

doi:10.1109/tmm.2022.3208516

Cited by 19 publications

(10 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning is a hot method in image processing in recent years, and researchers have constructed classifiers to achieve fast CU split by deep learning. In the literature [20], a three-level CNN network with early termination is designed and a CU partition map that can represent the three-level deep CU split is constructed as the output of the network, in the literature [21], a CNN convolutional neural network with asymmetric kernel is designed, a partition rate is proposed to make the network applicable to all quantization parameters, and the threshold decision of the network output is transformed into a multi-objective optimization problem to weigh the computational complexity and algorithm performance, in the literature [22,23], a Resnet model is proposed to predict the CTU partition of HEVC standard, in the literature [24], a fast decision algorithm for intra-frame modes based on convolutional neural networks is proposed by scaling prediction units of different sizes to CU of the same size by bilinear interpolation, followed by model learning to achieve fast decision making, in the literature [25] designed a three-level MSE-CNN network, which has a structure related to the kernel size and CU size, and designed an adaptive cross-entropy activation function to solve the problem of imbalance between different splitting cases, and literature [26] used a deep convolutional network to fuse all reference features obtained from varying convolutional kernels after extracting the spatio-temporal corresponding coding features to finally determine the intra-frame coding depth. And using probabilistic model and spatio-temporal coherence based to select the candidate split mode with the best encoding depth.…”

Section: Background and Related Workmentioning

confidence: 99%

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

2023

View full text Add to dashboard Cite

In order to satisfy the growing need for high-quality video, VVC comes with more efficient coding performance. According to statistical analysis, the level of coding complexity in VVC is tenfold greater compared to that of HEVC, so it is our main goal to study that what methods can be employed to decrease the time complexity of VVC. CU split in intra-frame modes requires the split mode decision by RD loss calculation, and the process of coding makes it to calculate RD calculation for all possible mode combinations, which is an important area that brings complexity to video coding, so in order to achieve our goal. Initially, we introduce an optimal depth prediction algorithm for Coding Units (CUs) by leveraging temporal combination. This algorithm collects depth information of CUs to predict the coding depth of CU blocks. Additionally, we suggest a decision tree-based method for CU split mode decision. With this method, we can make a decision on the CU split mode within the obtained split depth, reducing the time complexity of coding. This decision is based on the predictions from the first algorithm. The results demonstrate that our algorithm achieves superior performance over state-of-the-art methods in terms of computational complexity and compression quality. Compared to the VVC reference software (VTM), our method saves an average of 53.92% in coding time and improves the BDBR by 1.74%. These findings suggest that our method is highly effective in improving both computational efficiency and compression quality.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

2023

View full text Add to dashboard Cite

show abstract

“…The fast algorithms of H.266/VVC intra coding have been explored to solve the problem of the high computational requirement of H.266/VVC. The methods can be roughly categorized into probability-based [10][11][12], learning-based [13][14][15][16][17][18][19][20], probabilityand learning-based [21,22], texture-based [23][24][25][26], gradient-based [27,28], and texture-and gradient-based [29][30][31] techniques. The related work on the fast intra coding of H.266/VVC is discussed below.…”

Section: Related Workmentioning

confidence: 99%

Visual Perception Based Intra Coding Algorithm for H.266/VVC

Tsai

Chen

et al. 2023

Electronics

View full text Add to dashboard Cite

The latest international video coding standard, H.266/Versatile Video Coding (VVC), supports high-definition videos, with resolutions from 4 K to 8 K or even larger. It offers a higher compression ratio than its predecessor, H.265/High Efficiency Video Coding (HEVC). In addition to the quadtree partition structure of H.265/HEVC, the nested multi-type tree (MTT) structure of H.266/VVC provides more diverse splits through binary and ternary trees. It also includes many new coding tools, which tremendously increases the encoding complexity. This paper proposes a fast intra coding algorithm for H.266/VVC based on visual perception analysis. The algorithm applies the factor of average background luminance for just-noticeable-distortion to identify the visually distinguishable (VD) pixels within a coding unit (CU). We propose calculating the variances of the numbers of VD pixels in various MTT splits of a CU. Intra sub-partitions and matrix weighted intra prediction are turned off conditionally based on the variance of the four variances for MTT splits and a thresholding criterion. The fast horizontal/vertical splitting decisions for binary and ternary trees are proposed by utilizing random forest classifiers of machine learning techniques, which use the information of VD pixels and the quantization parameter. Experimental results show that the proposed algorithm achieves around 47.26% encoding time reduction with a Bjøntegaard Delta Bitrate (BDBR) of 1.535% on average under the All Intra configuration. Overall, this algorithm can significantly speed up H.266/VVC intra coding and outperform previous studies.

show abstract

“…Ref. [11] employs deep convolutional networks to extract spatiotemporal coding features by integrating reference features obtained from diverse convolutional kernels. These fused features are instrumental in determining the coding depth within a given frame.…”

Section: Status Of H266/vvc Research For 2d Videomentioning

confidence: 99%

Fast Algorithm for CU Size Decision Based on Ensemble Clustering for Intra Coding of VVC 3D Video Depth Map

Song

Zhang

2023

Electronics

View full text Add to dashboard Cite

As many new coding techniques and coding structures have been introduced to further improve the coding efficiency of depth maps in 3D video extensions, the coding complexity has been greatly increased. Fast algorithms are now needed to improve coding unit (CU) depth decisions as well as the coding pattern decision based on the coding. This paper presents an innovative machine learning-based approach aimed at mitigating the complexity associated with in-frame coding algorithms. We build different clustering models for different CU sizes to cluster CUs of the same size to decide their CU sizes. This is achieved by augmenting ensemble clustering through the expedited propagation of clustering similarities, considering CU with the same or similar texture complexity the same as for CU depth selection, which is informed by a comprehensive analysis of the original texture and its neighboring elements. The experimental findings demonstrate that the proposed scheme yields a substantial average reduction of 44.24% in the coding time. Remarkably, the corresponding Bjøntegaard delta bit rate (BDBR) increment observed for the synthetic view is a mere 0.26%.

show abstract

Efficient VVC Intra Prediction Based on Deep Feature Fusion and Probability Estimation

Cited by 19 publications

References 58 publications

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

Visual Perception Based Intra Coding Algorithm for H.266/VVC

Fast Algorithm for CU Size Decision Based on Ensemble Clustering for Intra Coding of VVC 3D Video Depth Map

Contact Info

Product

Resources

About