Probabilistic ToF and Stereo Data Fusion Based on Mixed Pixels Measurement Models

Mutto, Carlo Dal; Zanuttigh, Pietro; Cortelazzo, G.M.

doi:10.1109/tpami.2015.2408361

Cited by 33 publications

(38 citation statements)

References 46 publications

(69 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we propose to leverage a small set of sparse depth measurements to obtain, with deep stereo networks, dense and accurate estimations in any environment. It is worth pointing out that our proposal is different from depth fusion strategies (e.g., [17,21,5,1]) aimed at combining the output of active sensors and stereo algorithms such as Semi-Global Matching [10]. Indeed, such methods mostly aim at selecting the most reliable depth measurements from the multiple available using appropriate frameworks whereas our proposal has an entirely different goal.…”

Section: Introductionmentioning

confidence: 99%

Guided Stereo Matching

Poggi

Pallotti

Tosi

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

93.21% 4.36% (a) (b) (c) Figure 1. Guided stereo matching. (a) Challenging, reference image from KITTI 2015 [20] and disparity maps estimated by (b) iResNet [14] trained on synthetic data [19], or (c) guided by sparse depth measurements (5% density). Error rate (> 3) superimposed on each map. AbstractStereo is a prominent technique to infer dense depth maps from images, and deep learning further pushed forward the state-of-the-art, making end-to-end architectures unrivaled when enough data is available for training. However, deep networks suffer from significant drops in accuracy when dealing with new environments. Therefore, in this paper, we introduce Guided Stereo Matching, a novel paradigm leveraging a small amount of sparse, yet reliable depth measurements retrieved from an external source enabling to ameliorate this weakness. The additional sparse cues required by our method can be obtained with any strategy (e.g., a LiDAR) and used to enhance features linked to corresponding disparity hypotheses. Our formulation is general and fully differentiable, thus enabling to exploit the additional sparse inputs in pre-trained deep stereo networks as well as for training a new instance from scratch. Extensive experiments on three standard datasets and two stateof-the-art deep architectures show that even with a small set of sparse input cues, i) the proposed paradigm enables significant improvements to pre-trained networks. Moreover, ii) training from scratch notably increases accuracy and robustness to domain shifts. Finally, iii) it is suited and effective even with traditional stereo algorithms such as SGM.

show abstract

Section: Introductionmentioning

confidence: 99%

Guided Stereo Matching

Poggi

Pallotti

Tosi

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…In 2013, Engel et al [10] used the geometric disparity error and photometric disparity error for the structure from motion sensor to estimate 3D point error. Recently, many researchers [7,18] have estimated the uncertainty for the ToF (Time of Flight) sensor based on the physical properties of the sensor (eg. the IR frequency).…”

Section: Related Workmentioning

confidence: 99%

“…The challenges mainly lie in two parts: the first is how to get the real uncertainty distribution information from the real sensors. In the recent years, an increasing number of researchers have been investigating how to estimate the uncertainty of the acquired data for different sensors, such as the Kinect sensor [20], the time of flight sensor [7], the structure from motion sensor [10] and the stereo vision sensor [18]. These suggest using physical noise models for each point to represent their individual occurrence probability in 3D space.…”

Section: Introductionmentioning

confidence: 99%

DUGMA: Dynamic Uncertainty-Based Gaussian Mixture Alignment

Tyleček³

et al. 2018

2018 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

Accurately registering point clouds from a cheap lowresolution sensor is a challenging task. Existing rigid registration methods failed to use the physical 3D uncertainty distribution of each point from a real sensor in the dynamic alignment process. It is mainly because the uncertainty model for a point is static and invariant and it is hard to describe the change of these physical uncertainty models in different views. Additionally, the existing Gaussian mixture alignment architecture cannot efficiently implement these dynamic changes.This paper proposes a simple architecture combining error estimation from sample covariances and dynamic global probability alignment using the convolution of uncertaintybased Gaussian Mixture Models (GMM). Firstly, we propose an efficient way to describe the change of each 3D uncertainty model, which represents the structure of the point cloud better. Unlike the invariant GMM (representing a fixed point cloud) in traditional Gaussian mixture alignment, we use two uncertainty-based GMMs that change and interact with each other in each iteration. In order to have a wider basin of convergence than other local algorithms, we design a more robust energy function by convolving efficiently the two GMMs over the whole 3D space.Tens of thousands of trials have been conducted on hundreds of models from multiple datasets to demonstrate the proposed method's superior performance compared with the current state-of-the-art methods. All the materials including our code are available from https://github. com/Canpu999/DUGMA.

show abstract

“…Work by [19] proposed a reliable method by incorporating texture information, segmentation into a novel pseudo-two-layer model to improve the depth estimation. Work by [20] proposed a probabilistic method for fusing ToF data and stereo data based on mixed pixels measurement models. By using the complementary characteristics, these method show better results than the depth map by using the single sensor.…”

Section: Related Workmentioning

confidence: 99%