Bagged Tree and ResNet-Based Joint End-to-End Fast CTU Partition Decision Algorithm for Video Intra Coding

Abstract

Video coding standards, such as high-efficiency video coding (HEVC), versatile video coding (VVC), and AOMedia video 2 (AV2), achieve an optimal encoding performance by traversing all possible combinations of coding unit (CU) partition and selecting the combination with the minimum coding cost. It is still necessary to further reduce the encoding time of HEVC, because HEVC is one of the most widely used coding standards. In HEVC, the process of searching for the best performance is the source of most of the encoding complexity. To reduce the complexity of the coding block partition in HEVC, a new end-to-end fast algorithm is presented to aid the partition structure decisions of the coding tree unit (CTU) in intra coding. In the proposed method, the partition structure decision problem of a CTU is solved by a novel two-stage strategy. In the first stage, a bagged tree model is employed to predict the splitting of a CTU. In the second stage, the partition problem of a 32 × 32-sized CU is modeled as a 17-output classification task for the first time, so that it can be solved by a single prediction. To achieve a high prediction accuracy, a residual network (ResNet) with 34 layers is employed. Jointly using bagged tree and ResNet, the proposed fast CTU partition algorithm is able to generate the partition quad-tree structure of a CTU through an end-to-end prediction process, which abandons the traditional scheme of making multiple decisions at various depth levels. In addition, several datasets are used in this paper to lay the foundation for high prediction accuracy. Compared with the original HM16.7 encoder, the experimental results show that the proposed algorithm can reduce the encoding time by 60.29% on average, while the Bjøntegaard delta rate (BD-rate) loss is as low as 2.03%, which outperforms the results of most of the state-of-the-art approaches in the field of fast intra CU partition.

The implication of oversampling on the effectiveness of force signals in the fault detection of endodontic instruments during RCT

“…The bagging approach is introduced to address the ensemble bagged tree classifier (EBT). 40 Bagging is the term for bootstrap aggregation, an ensemble learning that generates data using sequential and parallel approaches. 41 In most cases, bagging uses a similar weak classifier and trains them simultaneously.…”

Fast CU Partition Algorithm for Intra Frame Coding Based on Joint Texture Classification and CNN

“…Recent studies have demonstrated that CNN-based algorithms [ 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 ] outperform traditional learn-based algorithms in terms of the predicting accuracy for the CU partition because they can automatically extract the features and classify them, and accelerate the speed of CU partition while maintaining high coding performance. Liu et al [ 21 ] first applied CNN to HEVC intra-frame CU partition and deployed it on hardware, thereby reducing encoder complexity.…”

“…Tahir et al [ 28 ] used a system based on the integration of online and offline random forest classifiers to segment CUs to accommodate the dynamic nature of video content. Li et al [ 29 ] used a new two-stage strategy for the CU fast partition decision. The CTU partition was predicted in the first stage using a bagged tree model.…”

Temporal Prediction Model-Based Fast Inter CU Partition for Versatile Video Coding

“…To reduce the coding complexity of the intra VVC encoder, many fast CU decision methods [ 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 ] have proposed the acceleration of intra QTMT partition. For example, in [ 19 ], the authors propose a directional gradient-based early termination CU partition method in which four directional gradients (horizontal, vertical, 45, and 135) of CUs are extracted separately to represent the relationship between the optimal CU partition and the texture characteristics of the CU.…”

“…In [ 23 ], a fast decision method for QTMT partition with a cascaded decision structure composed of decision trees is proposed. In addition, the authors in [ 25 , 26 , 27 , 28 , 29 ] adopted a deep-learning-based approach for fast QTMT partition decisions. However, these fast CU decision methods for intra QTMT partition cannot be directly used for inter QTMT partition, because the optimal intra CU partition of a CTU is mainly related to the texture characteristics of the CU under the intra coding.…”