Pseudo RGB-D for Self-improving Monocular SLAM and Depth Prediction

Tiwari, Lokender; Ji, Pan; Tran, Quoc-Huy; Zhuang, Bingbing; Anand, Sandeep; Chandraker, Manmohan

doi:10.1007/978-3-030-58621-8_26

Cited by 68 publications

(32 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposal is leveraging the depth predicted by a network to augment monocular images into what we call pseudo-RGBD views and then aligning the point clouds using ICP. Similar ideas were proposed recently in [28], [17]. Differently from us, they rely on Structure from Motion [27] or visual SLAM [20] to estimate the motion from the pseudo-RGBD views.…”

Section: Bayesian Pseudo-rgbd Icpmentioning

confidence: 92%

Bayesian Deep Neural Networks for Supervised Learning of Single-View Depth

Rodríguez-Puigvert¹,

Martínez-Cantín²,

Civera³

2021

Preprint

View full text Add to dashboard Cite

Uncertainty quantification is a key aspect in robotic perception, as overconfident or point estimators can lead to collisions and damages to the environment and the robot. In this paper, we evaluate scalable approaches to uncertainty quantification in single-view supervised depth learning, specifically MC dropout and deep ensembles. For MC dropout, in particular, we explore the effect of the dropout at different levels in the architecture. We demonstrate that adding dropout in the encoder leads to better results than adding it in the decoder, the latest being the usual approach in the literature for similar problems. We also propose the use of depth uncertainty in the application of pseudo-RGBD ICP and demonstrate its potential for improving the accuracy in such a task.

show abstract

Section: Bayesian Pseudo-rgbd Icpmentioning

confidence: 92%

Bayesian Deep Neural Networks for Supervised Learning of Single-View Depth

Rodríguez-Puigvert¹,

Martínez-Cantín²,

Civera³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The quantitative results on the KITTI Eigen test split are shown in Table 2 , where [ 20 , 28 , 30 , 44 , 69 , 71 , 72 ] adopted the same split strategy as our method. It is worth noting that Lee et al [ 20 ] devised a pyramid-like decoder with an encoder the same as ours, ResNeXt101.…”

Section: Methodsmentioning

confidence: 99%

DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Lai

Tian

et al. 2021

Sensors

View full text Add to dashboard Cite

Pyramid architecture is a useful strategy to fuse multi-scale features in deep monocular depth estimation approaches. However, most pyramid networks fuse features only within the adjacent stages in a pyramid structure. To take full advantage of the pyramid structure, inspired by the success of DenseNet, this paper presents DCPNet, a densely connected pyramid network that fuses multi-scale features from multiple stages of the pyramid structure. DCPNet not only performs feature fusion between the adjacent stages, but also non-adjacent stages. To fuse these features, we design a simple and effective dense connection module (DCM). In addition, we offer a new consideration of the common upscale operation in our approach. We believe DCPNet offers a more efficient way to fuse features from multiple scales in a pyramid-like network. We perform extensive experiments using both outdoor and indoor benchmark datasets (i.e., the KITTI and the NYU Depth V2 datasets) and DCPNet achieves the state-of-the-art results.

show abstract

“…Learning methods Deep learning has recently been applied to different problems in SFM, e.g. [45,47,58,55,78].…”

Section: Related Workmentioning

confidence: 99%

Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

Zhuang¹,

Chandraker²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Learning methods for relative camera pose estimation have been developed largely in isolation from classical geometric approaches. The question of how to integrate predictions from deep neural networks (DNNs) and solutions from geometric solvers, such as the 5-point algorithm [37], has as yet remained under-explored. In this paper, we present a novel framework that involves probabilistic fusion between the two families of predictions during network training, with a view to leveraging their complementary benefits in a learnable way. The fusion is achieved by learning the DNN uncertainty under explicit guidance by the geometric uncertainty, thereby learning to take into account the geometric solution in relation to the DNN prediction. Our network features a self-attention graph neural network, which drives the learning by enforcing strong interactions between different correspondences and potentially modeling complex relationships between points. We propose motion parmeterizations suitable for learning and show that our method achieves state-of-the-art performance on the challenging DeMoN [61] and ScanNet [8] datasets. While we focus on relative pose, we envision that our pipeline is broadly applicable for fusing classical geometry and deep learning.

show abstract

Pseudo RGB-D for Self-improving Monocular SLAM and Depth Prediction

Cited by 68 publications

References 58 publications

Bayesian Deep Neural Networks for Supervised Learning of Single-View Depth

Bayesian Deep Neural Networks for Supervised Learning of Single-View Depth

DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

Contact Info

Product

Resources

About