Joint upsampling and noise reduction for real-time depth map enhancement

Abstract

An efficient system that upsamples depth map captured by Microsoft Kinect while jointly reducing the effect of noise is presented. The upsampling is carried by detecting and exploiting the piecewise locally planar structures of the downsampled depth map, based on corresponding high-resolution RGB image. The amount of noise is reduced by accumulating the downsampled data simultaneously. By benefiting from massively parallel computing capability of modern commodity GPUs, the system is able to maintain high frame rate. Our system is observed to produce the upsampled depth map that is very close to the original depth map both visually and mathematically.

Illumination Estimation and Relighting using an RGB-D Camera

“…k ′ (u) is a normalize factor and Ω g ′ is a square window whose center is u. After applying bilateral filter, we also apply the temporal filter (Matsumoto et al, 2014). In our method, a current depth image is denoised by using a current frame and a previous frame.…”

A depth optimization method for 2D-to-3D conversion based on RGB-D images

“…According to Ref. [19], the surface normal at point (x 0 +n, y 0 +n) can be derived through solving the following optimization problem:…”

“…Some literatures make use of spatial and temporal information to fill holes in the original depth map [15][16]. Another research works apply segmentation and merging based on blocks to reduce the number of noisy pixels [17][18][19]. Most of the proposals have achieved good performance on removing small noisy regions and depth hole-filling on the set of Kinect depth images.…”