A method of hole-filling for the depth map generated by Kinect with moving objects detection

Xu, Kang; Zhou, Jun; Wang, Zhen

doi:10.1109/bmsb.2012.6264232

Cited by 10 publications

(8 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These depth hole regions do not need to be filled, because depth information of these regions can be acquired from other images that are captured at different viewpoints. [3], and the proposed methods, respectively. In Fig.…”

Section: Experimenal Results and Discussionmentioning

confidence: 99%

“…After that, depth map refinement proceeds by using the color image. Xu et al's method uses the maximum depth value in a window of a certain size to fill depth holes [3].…”

Section: Imentioning

confidence: 99%

“…After that, depth map refinement proceeds by using the color image. Xu et al's method uses the maximum depth value in a window of a certain size to fill depth holes [3].Recently, visible-light camera and range camera are combined to obtain color and depth information at the same time [4]. This type of camera is called RGBD camera, and the acquired information is called RGBD image.…”

mentioning

confidence: 99%

“…However, the depth image acquired by a range camera contains regions that do not have any depth information, which is called the depth hole problem. To solve the depth hole problem, various algorithms have been presented [2,3]. To fill holes, Yang et al's method uses the distribution of the depth values around depth holes [2].…”

mentioning

confidence: 99%

See 3 more Smart Citations

Depth hole filling for 3-D reconstruction using color and depth images

Kim

Piao

Kim

et al. 2014

The 18th IEEE International Symposium on Consumer Electronics (ISCE 2014)

View full text Add to dashboard Cite

Computer vision algorithms using depth and color data (RGBD image) acquired at the same time are gradually getting an attention. Depth data is useful three-dimensional (3-D) information. However, the depth data has a problem that it includes depth hole regions. In this paper, a depth hole filling method is proposed, which is suitable for 3-D reconstruction, with the detail of depth region of an object preserved. Keywords-depth hole; 3-D reconstruction; RGBD image; depth map enhancement I.INTRODUCTION Three-dimensional (3-D) information is obtained by using a range camera. The early type of range camera is composed of radio frequency (RF) modulated light sources and the receiver that includes phase detector [1]. To measure the distance between an object and a camera, this type of range camera emits RF-modulated light. Then, the receiver of a range camera measures the distance by detecting phase difference of RFmodulated light. Another type of range camera consists of structured infrared light (SIL) source and infrared camera [1]. SIL source emits regular patterns of infrared light into scene, then infrared camera produces image by detecting emitted patterns of infrared light. The distance between a range camera and an object is measured by observing patterns in the image. However, the depth image acquired by a range camera contains regions that do not have any depth information, which is called the depth hole problem. To solve the depth hole problem, various algorithms have been presented [2,3]. To fill holes, Yang et al.'s method uses the distribution of the depth values around depth holes [2]. The value that fills depth holes is computed by averaging peak values of the distribution of the depth values around depth holes. After that, depth map refinement proceeds by using the color image. Xu et al.'s method uses the maximum depth value in a window of a certain size to fill depth holes [3].Recently, visible-light camera and range camera are combined to obtain color and depth information at the same time [4]. This type of camera is called RGBD camera, and the acquired information is called RGBD image. RGBD image is very useful information because it represents not only color information but also 3-D information. However, RGBD image has the depth hole problem, which degrades the performance. For example, the depth hole problem is a critical factor in 3-D reconstruction. In [5], Lim et al. use scale invariant feature transform (SIFT) features to match multiple RGBD images acquired at different viewpoints. Most of SIFT features are located at edges of RGB image. The depth holes also lie at edges of RGB image. Thus, most of SIFT features do not have

show abstract

Section: Experimenal Results and Discussionmentioning

confidence: 99%

“…After that, depth map refinement proceeds by using the color image. Xu et al's method uses the maximum depth value in a window of a certain size to fill depth holes [3].…”

Section: Imentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Depth hole filling for 3-D reconstruction using color and depth images

Kim

Piao

Kim

et al. 2014

The 18th IEEE International Symposium on Consumer Electronics (ISCE 2014)

View full text Add to dashboard Cite

show abstract

“…Liu et al [3] proposed a spatial inpainting method based on the extended fast matching method, which incorporates an aligned color image as the guidance for inpainting and applies an edge-preserving guided filter for noise reduction. Xu et al [4] proposed a hole-filling method with motion objects detection for Kinect depth image which used 4-neighbor-pixels-interpolation to the none-body areas before filling the body areas. This method can fill holes while remaining the edges' information.…”

Section: Introductionmentioning

confidence: 99%

A spatio-temporal inpainting method for Kinect depth video

Zhang

Yao

Zang

et al. 2013

2013 IEEE International Conference on Signal and Image Processing Applications

View full text Add to dashboard Cite

In this paper, we propose a spatio-temporal inpainting method to recover depth images generated by Kinect. Based on the assumption that neighbouring pixels similar in color are likely to have similar depth values, for the first depth image and the sub-images including the motion bodies extracted from the following depth frames, we use color segmentation maps of the corresponding color video frames to guide the depth filling. Considering that some dark regions without valid depth value could lead to the fail of color-segmentation based depth filling, we design a dark region detection method and further refine hole-filling of the unfilled regions with the valid depth values of the same dark region. For the static areas of Kinect depth video, the recovered depth at the same position of previous frame is used to recover the lost depth in the current depth frame. Experimental results show that the proposed method significantly improves depth quality by successfully filling the holes so that we can use it for better 3D rendering.

show abstract