Joint processing and fast encoding algorithm for multi-view depth video

Abstract

The multi-view video plus depth format is the main representation of a three-dimensional (3D) scene. In the 3D extension of high-efficiency video coding (3D-HEVC), the main framework for depth video is similar to that of color video. However, because of the limitation of the mainstream capture technologies, depth video is inaccurate and inconsistent. In addition, the depth video coding method in the current 3D-HEVC software implementation is highly complex. In this paper, we introduce a joint processing and fast coding algorithm for depth video. The proposed algorithm utilizes the depth and color features to extract depth discontinuous regions, depth edge regions, and motion regions as masks for efficient processing and fast coding. The processing step includes spatial and temporal enhancement. The fast coding method mainly limits the traversal of the CU partition and mode decision. Experimental results demonstrate that the proposed algorithm reduces the coding time and the depth video coding bitrate; the proposed algorithm reduces overall coding time by 44.24 % and depth video coding time by 72.00 % on average. In addition, there is a 24.07 % depth video coding bitrate reduction with an average of 1.65 % Bjontegaard delta bitrate gains.

Highly Efficient Multiview Depth Coding Based on Histogram Projection and Allowable Depth Distortion

“…To further improve the coding efficiency, a number of depth coding algorithms [12]- [20] were proposed by further exploiting depth map properties. Peng et al [12] proposed spatial and temporal enhancement filters for the depth discontinuous regions, depth edge regions, and motion regions, which reduce the prediction residual error and coding complexity in mode decision.…”

“…To further improve the coding efficiency, a number of depth coding algorithms [12]- [20] were proposed by further exploiting depth map properties. Peng et al [12] proposed spatial and temporal enhancement filters for the depth discontinuous regions, depth edge regions, and motion regions, which reduce the prediction residual error and coding complexity in mode decision. Since the depth edge is of greater importance, Shahriyar et al [13] proposed a mono-view depth encoder, which preserves edges implicitly by limiting quantization to the spatial-domain.…”

Fast and adaptive mode decision and CU partition early termination algorithm for intra-prediction in HEVC

“…The parallel deblocking filter for HEVC was proposed by Yan et al [14]; they decrease HEVC complexity in decoder process. Peng et al [15] introduced a fast coding algorithm that uses depth and color features to extract depth discontinuous regions, depth edge regions, and motion regions as masks for efficient processing and fast coding. Nishikori et al [16] of the Department of Information and Electronics of the Tottori University in Japan proposed a fast CU size selection method that determined CU sizes using the variance value of the input image.…”

“…Cen et al [22] introduced a fast adaptive CU depth decision mechanism that utilizes the special correlations in the sequence frame, saving average time of 16% compared with HM 10.0. The aforementioned papers [15][16][17][18][19][20][21][22] are fast algorithms aimed at speeding up the HEVC intra-prediction encoder through early termination of CU partition. There are other algorithms aimed to optimize HEVC in different way, for example, Yan et al [23] proposed a parallel framework to decide coding unit trees on many-core processors.…”