BLINC: lightweight bimodal learning for low-complexity VVC intra-coding

Pakdaman, Farhad; Adelimanesh, Mohammad Ali; Hashemi, Mahmoud Reza

doi:10.1007/s11554-022-01223-1

Cited by 2 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mercat et al [5] introduced a classification algorithm for QTBT partitioning, and Wu et al [27] proposed an algorithm with an RF classifier to classify CUs, predicting the optimal splitting mode for CUs. Most prominently used in VVC are CNN [3], [28], [29], [30] networks, and the lightweight LNN [31], [32], [33] networks. Xu et al [28] proposed a multistage early-exit CNN model (MSE-CNN) to determine CU partitions using an early-exit mechanism.…”

Section: B Vvc Methodsmentioning

confidence: 99%

“…The model proposed by Park et al [31] decides whether to terminate nested structures based on features. Pakdaman et al [32] used two feature patterns to work together for internal coding decisions Chen et al [33] proposed an LFHI framework that uses AK-CNN to predict the optimal number of candidates, which has high parallelism and can be generalized to all quantization parameters.…”

Section: B Vvc Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Fast CU Partitioning Algorithm for VVC Based on CNN and FSVM

Song,

LI,

Zhang

2024

IEEE Access

View full text Add to dashboard Cite

The new coding standard Versatile Video Coding (VVC) introduces additional encoding techniques based on the existing video coding standard, such as the difference in block partition structures. While these new technologies bring about enhanced encoding performance, they also result in a significant increase in encoding time complexity. In the paper, we propose a decision algorithm to partition fast CUs, which is based on CNN networks and FSVM. Initially, the algorithm utilizes depth information formed by combining inter-frame correlations as input to our trained CNN model, predicting the optimal depth for encoding.Following that, the second algorithm based on FSVM is introduced. The F-Score method is employed to extract appropriate features for constructing FSVM. Within the predicted depth range from the initial algorithm, the CU partitioning mode is predicted, leading to an additional reduction in encoding complexity. Our experimental results demonstrate that the proposed algorithm can save 53.55% of encoding time, with a marginal increase of only 1.47% in BDBR.It achieves a favorable balance between video quality and encoding speed.

show abstract

Section: B Vvc Methodsmentioning

confidence: 99%

Section: B Vvc Methodsmentioning

confidence: 99%

Fast CU Partitioning Algorithm for VVC Based on CNN and FSVM

Song,

LI,

Zhang

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

A Fast Coding Unit Partitioning Decision Algorithm for Versatile Video Coding Based on Gradient Feedback Hierarchical Convolutional Neural Network and Light Gradient Boosting Machine Decision Tree

Liu,

Wang,

Zhang

2024

Electronics

View full text Add to dashboard Cite

Video encoding technology is a foundational component in the advancement of modern technological applications. The latest standard in universal video coding, H.266/VVC, features a quad-tree with nested multi-type tree (QTMT) partitioning structure, which represents an improvement over its predecessor, High-Efficiency Video Coding (H.265/HEVC). This configuration facilitates adaptable block segmentation, albeit at the cost of heightened encoding complexity. In view of the aforementioned considerations, this paper puts forth a deep learning-based approach to facilitate CU partitioning, with the aim of supplanting the intricate CU partitioning process observed in the Versatile Video Coding Test Model (VTM). We begin by presenting the Gradient Feedback Hierarchical CNN (GFH-CNN) model, an advanced convolutional neural network derived from the ResNet architecture, enabling the extraction of features from 64 × 64 coding unit (CU) blocks. Following this, a hierarchical network diagram (HND) is crafted to depict the delineation of partition boundaries corresponding to the various levels of the CU block’s layered structure. This diagram maps the features extracted by the GFH-CNN model to the partitioning at each level and boundary. In conclusion, a LightGBM-based decision tree classification model (L-DT) is constructed to predict the corresponding partition structure based on the prediction vector output from the GFH-CNN model. Subsequently, any errors in the partitioning results are corrected in accordance with the encoding constraints specified by the VTM, which ultimately determines the final CU block partitioning. The experimental results demonstrate that, in comparison with VTM-10.0, the proposed algorithm achieves a 48.14% reduction in complexity with only a 0.83% increase in bitrate under the top-three configuration, which is negligible. In comparison, the top-two configuration resulted in a higher complexity reduction of 63.78%, although this was accompanied by a 2.08% increase in bitrate. These results demonstrate that, in comparison to existing solutions, our approach provides an optimal balance between encoding efficiency and computational complexity.

show abstract

BLINC: lightweight bimodal learning for low-complexity VVC intra-coding

Cited by 2 publications

References 36 publications

Fast CU Partitioning Algorithm for VVC Based on CNN and FSVM

Fast CU Partitioning Algorithm for VVC Based on CNN and FSVM

A Fast Coding Unit Partitioning Decision Algorithm for Versatile Video Coding Based on Gradient Feedback Hierarchical Convolutional Neural Network and Light Gradient Boosting Machine Decision Tree

Contact Info

Product

Resources

About