Methods for depth-map filtering in view-plus-depth 3D video representation

Abstract

View-plus-depth is a scene representation format where each pixel of a color image or video frame is augmented by per-pixel depth represented as gray-scale image (map). In the representation, the quality of the depth map plays a crucial role as it determines the quality of the rendered views. Among the artifacts in the received depth map, the compression artifacts are usually most pronounced and considered most annoying. In this article, we study the problem of post-processing of depth maps degraded by improper estimation or by block-transformbased compression. A number of post-filtering methods are studied, modified and compared for their applicability to the task of depth map restoration and post-filtering. The methods range from simple and trivial Gaussian smoothing, to in-loop deblocking filter standardized in H.264 video coding standard, to more comprehensive methods which utilize structural and color information from the accompanying color image frame. The latter group contains our modification of the powerful local polynomial approximation, the popular bilateral filter, and an extension of it, originally suggested for depth super-resolution. We further modify this latter approach by developing an efficient implementation of it. We present experimental results demonstrating high-quality filtered depth maps and offering practitioners options for highest-quality or better efficiency.

Using depth maps to find interesting regions

“…Depth quantization is normally done in linear or logarithmic scale. The latter approach allows better preservation of geometry details for closer objects, while higher geometry degradation is tolerated for objects at longer distances [2]. This paper uses relative depth as the primary selection criterion to identify interesting regions.…”

“…It is usually converted to integer values (e.g., in 256 gray-scale gradations). Ideally, the depth range and resolution should be properly maintained by suitable scaling, shifting, and quantizing [2]. These transformations should be invertible.…”

Fixed-pattern noise suppression in low-sensing environment of Time-of-Flight devices

“…We quantify the performance of several modern denoising algorithms [14,15,16,17,18], specifically selected since they have been used in similar applications [7,17,19,20,21]. Our experiments show that the performance of those methods is highly prone to artifacts caused by FPN.…”

A deblocking algorithm based on color psychology for display quality enhancement

“…Blocking artifacts exists both in luminance and chrominance components; however, most of the conventional schemes only focus on luminance for deblocking process. Not only in luminance, the study in [19] removes the blocking artifacts both in luminance and chrominance components and obtains better results. The proposed algorithm removes blocking artifacts in the RGB domain instead of YCbCr; therefore, it can simultaneously eliminate blocking artifacts from two components.…”

“…The low-pass filter is also employed to eliminate blocking artifacts in video compression standards [19]. Tai et al [20] proposed a deblocking algorithm based on five filtering modes by considering the activity across block boundaries.…”