Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding

Abstract

In this paper, we propose a machine learning-based fast coding unit (CU) depth decision method for High Efficiency Video Coding (HEVC), which optimizes the complexity allocation at CU level with given rate-distortion (RD) cost constraints. First, we analyze quad-tree CU depth decision process in HEVC and model it as a three-level of hierarchical binary decision problem. Second, a flexible CU depth decision structure is presented, which allows the performances of each CU depth decision be smoothly transferred between the coding complexity and RD performance. Then, a three-output joint classifier consists of multiple binary classifiers with different parameters is designed to control the risk of false prediction. Finally, a sophisticated RD-complexity model is derived to determine the optimal parameters for the joint classifier, which is capable of minimizing the complexity in each CU depth at given RD degradation constraints. Comparative experiments over various sequences show that the proposed CU depth decision algorithm can reduce the computational complexity from 28.82% to 70.93%, and 51.45% on average when compared with the original HEVC test model. The Bjøntegaard delta peak signal-to-noise ratio and Bjøntegaard delta bit rate are -0.061 dB and 1.98% on average, which is negligible. The overall performance of the proposed algorithm outperforms those of the state-of-the-art schemes.

Probabilistic Decision Based Block Partitioning for Future Video Coding

“…Shen et al suggested a fast Bayesian theory based CU decision algorithm [36], where the sum of absolute transform differences (SATD), MV and RD costs were employed as the features. In [37], a three-output joint classifier was designed to control the risk of false prediction. In the machine learning based methods, almost all the information can be utilized to make the decision, such as the encoding parameters, intermediate values, spatio-temporal neighbouring blocks, texture complexities, and the performance of decision structure strongly relies on the selected features.…”

Fast Intra-Mode and CU Size Decision for HEVC

“…SVM [28] has been used in fast inter mode decision for HEVC [29] [30]. In this subsection, we model the CU size decision as a binary classification problem.…”

Inter Prediction Complexity Reduction for HEVC based on Residuals Characteristics

“…In [41], the quad-tree CU depth decision process is modeled as a threelevel of hierarchical binary decision problem. In addition, a flexible CU depth decision structure is used to allow the performance of each CU depth decision be smoothly transferred between the coding complexity and RD performance along with binary classifiers to control the risk of false prediction.…”