Fusion of active and passive sensors for fast 3D capture

Yang, Qingxiong; Tan, Kar-Han; Culbertson, Bruce; Apostolopoulos, John

doi:10.1109/mmsp.2010.5661996

Cited by 62 publications

(44 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another approach was proposed in [7] where the depth map acquired by the ToF is reprojected on the reference image of the stereo pair, it is then interpolated and finally used as initialization for the application of a stereo vision algorithm. In [8] after the upsampling of the depth map acquired by the ToF by a hierarchical application of bilateral filtering, the authors apply a plane-sweeping stereo algorithm and finally a confidence based strategy is used for data fusion. In [9] the final depth map is recovered from the one acquired by the ToF and the one estimated with the stereo vision system by performing a MAP local optimization in order to increase the accuracy of the depth measurements.…”

Section: Introductionmentioning

confidence: 99%

Locally Consistent ToF and Stereo Data Fusion

Mutto

Zanuttigh

Mattoccia

et al. 2012

Computer Vision – ECCV 2012. Workshops and Demonstrations

View full text Add to dashboard Cite

Abstract. Depth estimation for dynamic scenes is a challenging and relevant problem in computer vision. Although this problem can be tackled by means of ToF cameras or stereo vision systems, each of the two systems alone has its own limitations. In this paper a framework for the fusion of 3D data produced by a ToF camera and a stereo vision system is proposed. Initially, depth data acquired by the ToF camera are up-sampled to the spatial resolution of the stereo vision images by a novel up-sampling algorithm based on image segmentation and bilateral filtering. In parallel a dense disparity field is obtained by a stereo vision algorithm. Finally, the up-sampled ToF depth data and the disparity field provided by stereo vision are synergically fused by enforcing the local consistency of depth data. The depth information obtained with the proposed framework is characterized by the high resolution of the stereo vision system and by an improved accuracy with respect to the one produced by both subsystems. Experimental results clearly show how the proposed method is able to outperform the compared fusion algorithms.

show abstract

Section: Introductionmentioning

confidence: 99%

Locally Consistent ToF and Stereo Data Fusion

Mutto

Zanuttigh

Mattoccia

et al. 2012

Computer Vision – ECCV 2012. Workshops and Demonstrations

View full text Add to dashboard Cite

show abstract

“…In a sense that the WMF computes the solution by finding the maximum on the joint histogram, the 3-D JBU and the WMF use a similar principle. For finding a maximum value on the 3-D cost volume, we redefine cost function as follows: (12) After applying the same joint bilateral filtering in (10), an output depth value, which is a maximum value on the 3-D cost volume, is the same to the solution in (11). If we assume that the new cost function is an approximated one of Gaussian function in (5), 3-D cost volume in (10) plays a similar role to joint histogram in (5), except that is computed after the normalization step.…”

Section: ) 2-d Jbumentioning

confidence: 99%

“…The iterative bilateral filtering on the cost domain results in better edge-preserving performance, but its computational complexity is times of that of the 2-D JBU [10], where is the number of depth candidates. Hierarchical depth upsampling [12] was proposed for an efficient implementation, but the complexity is still high and dependent on the number of depth candidates. In this paper, we call the JBU by Kopf et al [10] 2-D JBU, and the approach of Yang et al [11] 3-D JBU.…”

mentioning

confidence: 99%

Depth Video Enhancement Based on Weighted Mode Filtering

Min

2012

IEEE Trans. on Image Process.

210

View full text Add to dashboard Cite

Abstract-This paper presents a novel approach for depth video enhancement. Given a high-resolution color video and its corresponding low-quality depth video, we improve the quality of the depth video by increasing its resolution and suppressing noise. For that, a weighted mode filtering method is proposed based on a joint histogram. When the histogram is generated, the weight based on color similarity between reference and neighboring pixels on the color image is computed and then used for counting each bin on the joint histogram of the depth map. A final solution is determined by seeking a global mode on the histogram. We show that the proposed method provides the optimal solution with respect to 1 norm minimization. For temporally consistent estimate on depth video, we extend this method into temporally neighboring frames. Simple optical flow estimation and patch similarity measure are used for obtaining the high-quality depth video in an efficient manner. Experimental results show that the proposed method has outstanding performance and is very efficient, compared with existing methods. We also show that the temporally consistent enhancement of depth video addresses a flickering problem and improves the accuracy of depth video.Index Terms-Depth enhancement, depth sensor, multiscale color measure (MCM), temporal consistency, weighted mode filtering (WMF).

show abstract

“…Most of the works proposed to enhance the resolution and quality of depth images have been based on fusion with a high resolution (HR) image acquired with a second camera, e.g., a 2-D camera [4,5], a stereo camera [6], or both 2-D and stereo cameras [7]. These multi-modality methods suffer from drawbacks such as undesired texture copying, and blurring artifacts.…”

Section: Introductionmentioning

confidence: 99%

Enhancement of dynamic depth scenes by upsampling for precise super-resolution (UP-SR)

Ismaeil

Aouada

Mirbach

et al. 2016

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Multi-frame super-resolution is the process of recovering a high resolution image or video from a set of captured low resolution images. Super-resolution approaches have been largely explored in 2-D imaging. However, their extension to depth videos is not straightforward due to the textureless nature of depth data, and to their high frequency contents coupled with fast motion artifacts. Recently, few attempts have been introduced where only the super-resolution of static depth scenes has been addressed. In this work, we propose to enhance the resolution of dynamic depth videos with non-rigidly moving objects. The proposed approach is based on a new data model that uses densely upsampled, and cumulatively registered versions of the observed low resolution depth frames. We show the impact of upsampling in increasing the sub-pixel accuracy and reducing the rounding error of the motion vectors. Furthermore, with the proposed cumulative motion estimation, a high registration accuracy is achieved between non-successive upsampled frames with relative large motions. A statistical performance analysis is derived in terms of mean square error explaining the effect of the number of observed frames and the effect of the super-resolution factor at a given noise level. We evaluate the accuracy of the proposed algorithm theoretically and experimentally as function of the SR factor, and the level of contaminations with noise. Experimental results on both real and synthetic data show the effectiveness of the proposed algorithm on dynamic depth videos as compared to state-of-art methods.

show abstract

Fusion of active and passive sensors for fast 3D capture

Cited by 62 publications

References 13 publications

Locally Consistent ToF and Stereo Data Fusion

Locally Consistent ToF and Stereo Data Fusion

Depth Video Enhancement Based on Weighted Mode Filtering

Enhancement of dynamic depth scenes by upsampling for precise super-resolution (UP-SR)

Contact Info

Product

Resources

About