Study of 3D Video Compression Using Nonlinear Depth Representation

Abstract

The extensions of Advanced Video Coding (AVC) and High-Efficiency Video Coding (HEVC) for multiview video plus depth allow for the coding of 3D video scenes. During the standardization of these extensions, it was demonstrated that compression performance can often be improved by the application of nonlinear depth transformation prior to the compression and applying the respective backward nonlinear depth transformation after decompression. Such an approach is referred to as the compression of nonlinear depth representation (NDR). In this paper, a survey of the NDR technology is provided, but a large part of this survey comprises yet unpublished results. First, the motivation and rationale behind the idea of NDR are discussed. The options for the choice of nonlinear transformation are provided with the respective formulations. Also, the standardization of 3D video coding with NDR within ISO/IEC and ITU is discussed along with the experimental results that demonstrate benefits related to the application of NDR in 3D-AVC and 3D-HEVC. Furthermore, the experimental comparisons are provided for simulcast coding of multiview plus depth video using AVC and HEVC technology with and without the NDR. The similar experimental results are also provided for simulcast coding using the current version of Versatile Video Coding technology that is under development for the forthcoming standard of ISO/IEC (MPEG-I) and ITU. All of the experimental results have been obtained for the standard JVC-3V video test sequences under common test conditions for 3D video coding.

Highly Efficient Multiview Depth Coding Based on Histogram Projection and Allowable Depth Distortion

“…Third, in addition to MV-HEVC and 3D-HEVC, we also encoded the original and processed depth maps with other lossless coding standards for static images, including JPEG2000 and HEVC Intra coding, to test the coding efficiency of the proposed methods. Three benchmark schemes, including NDR [18] and the scheme in [44] for preprocessing, Intra depth wedge plus intra contour scheme [4] (denoted as DWC) for coding optimization, were implemented and compared on different reference platforms. Ten standard 3D sequences Bookarrival, Alt-Moabit, Lovebird1, Kendo, Balloons, Newspapers, PoznanCapark, PoznanHall2, PoznanStreet and Undodancers, were used in the coding experiments.…”

“…To improve the quality of distorted multiview depth maps in asymmetric coding, residual learning framework [15], Convolutional Neural Network (CNN) based up-sampling [16] and cross-view multi-lateral filter [17] were proposed to enhance the up-sampling quality of depth maps, where the correlations among viewpoints and between color and depth channels were exploited. Stankiewicz et al [18] proposed 3D depth coding algorithms using Nonlinear Depth Representation (NDR), where a power law transformation and a piece-wise function were used to nonlinearly remap the depth information according to its relative importance. For example, closer objects were given a higher dynamic range.…”

Block Based Enhancement using Deep Learning for Conversion of Low Resolution AVS Video to High Resolution HEVC Video

“…Chia-Hung et al [1] proposed a coding unit complexity based predictions of coding unit depth and prediction unit mode for efficient HEVC to SHVC transcoding with quality scalability. A survey of video compression is also given in [22].…”

Simultaneous Stream Transmission Methods for Free Viewpoint TV: A Comparative Study

“…The area of viewpoint rendering has attracted a lot of research interest. The basic idea behind IBR techniques is seven-dimensional plenoptic function [25]. It illustrates all available optical information in a given region.…”