The performance of the latest video coding standard, high efficiency video coding (HEVC), has been improved significantly compared with that of H.264/AVC. However, the heuristic rate-distortion optimization process (RDO) increases noticeable computational complexity in HEVC intra coding as well as inter coding. In this paper, a fast intra CU depth decision algorithm based on support vector machine (SVM) is proposed.Five features, including the depth of neighboring CUs (left and upper), average pixel difference on CU boundary (left and upper), pixel variance of current CU, variance of the mean of sub-CUs, and number of edge points, were selected for training SVM models. To enhance the prediction of SVM, this work used artificial neural network (ANN) to investigate how the five features could impact on classifications. The analysis results showed that the variance-type features were the most dominant factor while the number of edge points only had obvious impact at depth 0. Hence the edge feature was excluded from SVM model training. The rest of features were classified into 3 types, i.e. depthtype, difference-type, and variance-type, for 3 different SVM models. According to the ANN analysis, the prediction results of depth-type, difference-type, and variance-type SVM models were assigned weights with 0.2, 0.2, and 0.6 respectively to form the final SVM result (Result All ). In addition, the influence of each feature was different at each depth. This work applied distinct SVM models to each depth to improve the prediction accuracy. Moreover, a threshold was set for edge point (EP) to determine whether depth 0 should be reserved for RDO process, even if the final result of SVM was early splitting. The edge point checking was helpful to keep the background region to be encoded with large CU size as the original best partition. The intra CU depth decision was made based on Table I. Experimental results showed that the proposed algorithm can approximately achieve 46.5% time saving on average with 2.2% Bjontegaard delta (BD) bitrate loss compared with HM 12.1. TABLE I Proposed fast CU depth decision Condition Operation Result All 0.2 Early Termination Do RDO process at current depth and no further splitting 0.2 < Result All < 1 Unsure Original HEVC encoding Result All = 1 && Depth = 0 && EP < 40 Result All = 1 Early Splitting
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.