Novel distortion metric for depth coding of 3D video

Abstract

In state-of-the-art HEVC-based 3D video codec, multiview video plus associated depth maps are used. In order to achieve better coding performance, instead of the conventional sum of squared errors (SSE), view synthesis optimization (VSO) is proposed and included in the anchor encoder software to calculate view synthesis distortion in rate-distortion optimization (RDO) of depth coding. The anchor VSO achieves high rate-distortion (RD) performance. However, it requires partial rendering and is quite complex and time-consuming. On the other hand, simple SSE metric is fast but RD performance is low. In this paper, we propose a new distortion metric to be used in RDO for depth coding. The complexity of the proposed method is slightly higher than SSE, while its RD performance remains competitive. With a good trade-off between complexity and performance, the proposed method can replace the conventional SSE metric in RDO for depth coding, and can be used as a low-complexity alternative to the anchor VSO. IEEE International Conference on Image Processing (ICIP)This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. ABSTRACTIn state-of-the-art HEVC-based 3D video codec, multiview video plus associated depth maps are used. In order to achieve better coding performance, instead of the conventional sum of squared errors (SSE), view synthesis optimization (VSO) is proposed and included in the anchor encoder software to calculate view synthesis distortion in rate-distortion optimization (RDO) of depth coding. The anchor VSO achieves high rate-distortion (RD) performance. However, it requires partial rendering and is quite complex and time-consuming. On the other hand, simple SSE metric is fast but RD performance is low. In this paper, we propose a new distortion metric to be used in RDO for depth coding. The complexity of the proposed method is slightly higher than SSE, while its RD performance remains competitive. With a good trade-off between complexity and performance, the proposed method can replace the conventional SSE metric in RDO for depth coding, and can be used as a low-complexity alternative to the anchor VSO.

Optimized Data Representation for Interactive Multiview Navigation

“…Modeling process: distortion model We next investigate the modeling of the distortion term U D (V, S 0 ) in the navigation problem of (3). We first look at the view synthesis distortion at a single viewpoint, namely u D (r; V, S), which is in general difficult to estimate [34]. However, under the allocation solution S 0 , we can derive…”

Segment-based view synthesis optimization scheme in 3D-HEVC

“…Many researchers have been devoted their efforts on reducing the complexity of VSO [18][19][20][21][22][23][24][25][26][27].…”

“…Since distortions in synthesized views are caused by the depth map distortions due to the encoding process, some algorithms were proposed by deriving relationships between distortions in the coded depth map and rendered virtual view [18][19][20][21]. The well-known technique is called as view synthesis distortion (VSD) [19], which is accepted as part of the 3D-HEVC reference software HTM-16.0 [17].…”

“…However, these algorithms [18][19][20][21][22][23][24] estimate the distortion in synthesized views using mathematical models or metrics. Their results are approximate values and may not represent the accurate synthesized view distortion.…”