UnDEMoN: Unsupervised Deep Network for Depth and Ego-Motion Estimation

Babu, Vemuri Venkata Phani; Das, Kaushik; Majumdar, Anima; Kumar, Swagat

doi:10.1109/iros.2018.8593864

Cited by 23 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised learning of depth with stereo pairs based on novel view synthesis [10] that uses an image reconstruction loss in which the predicted depth is used for warping one image of the pair to the frame of the other was introduced in [16] and was cast in a fully differentiable formulation in [18]. Further works in this direction leverage temporal information [47,52,82]. The need for stereo pairs in this framework was lifted in [85], which operates on monocular videos.…”

Section: Related Workmentioning

confidence: 99%

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Patil¹,

Sakaridis²,

Liniger³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Monocular depth estimation is vital for scene understanding and downstream tasks. We focus on the supervised setup, in which ground-truth depth is available only at training time. Based on knowledge about the high regularity of real 3D scenes, we propose a method that learns to selectively leverage information from coplanar pixels to improve the predicted depth. In particular, we introduce a piecewise planarity prior which states that for each pixel, there is a seed pixel which shares the same planar 3D surface with the former. Motivated by this prior, we design a network with two heads. The first head outputs pixel-level plane coefficients, while the second one outputs a dense offset vector field that identifies the positions of seed pixels. The plane coefficients of seed pixels are then used to predict depth at each position. The resulting prediction is adaptively fused with the initial prediction from the first head via a learned confidence to account for potential deviations from precise local planarity. The entire architecture is trained end-to-end thanks to the differentiability of the proposed modules and it learns to predict regular depth maps, with sharp edges at occlusion boundaries. An extensive evaluation of our method shows that we set the new state of the art in supervised monocular depth estimation, surpassing prior methods on NYU Depth-v2 and on the Garg split of KITTI. Our method delivers depth maps that yield plausible 3D reconstructions of the input scenes. Code is available at: https://github.com/SysCV/P3Depth

show abstract

Section: Related Workmentioning

confidence: 99%

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Patil¹,

Sakaridis²,

Liniger³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…In their work, the four fundamental vision problems are solved simultaneously through geometric constraints. [Babu et al 2018;Zhan et al 2018] adopted temporal information in optimisation. These methods have all made use of consecutive video frames to train networks (i.e.…”

Section: Stereo Depth Estimationmentioning

confidence: 99%

Constant Velocity Constraints for Self-Supervised Monocular Depth Estimation

Zhou

Greenwood

Taylor

et al. 2020

European Conference on Visual Media Production

View full text Add to dashboard Cite

“…Depth regularization Discontinuity of depth usually happens where strong image gradients are present. Similar to [4,40], we introduce an edge-aware smoothness loss to enforce discontinuity and local smoothness in depth…”

Section: Loss Functionsmentioning

confidence: 99%

“…As the first selfsupervised approach for VO, SfMLearner couples depth and pose estimations with image warping, which becomes the problem of minimizing photometric loss. Inherited from this idea, many self-supervised VO have been proposed, including modifications on loss functions [22,26], network architectures [3,4,22,28,40], predicted contents [39], and combination with classic VO/SLAM [5,38]. For example, GeoNet [39] extends the framework to jointly estimate optical flow with forward-backward consistency to infer unstable regions and achieves state-of-the-art performance among self-supervised VO methods.…”

Section: Related Workmentioning

confidence: 99%

Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry

Xue

Wang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We propose a self-supervised learning framework for visual odometry (VO) that incorporates correlation of consecutive frames and takes advantage of adversarial learning. Previous methods tackle self-supervised VO as a local structure from motion (SfM) problem that recovers depth from single image and relative poses from image pairs by minimizing photometric loss between warped and captured images. As single-view depth estimation is an ill-posed problem, and photometric loss is incapable of discriminating distortion artifacts of warped images, the estimated depth is vague and pose is inaccurate. In contrast to previous methods, our framework learns a compact representation of frame-to-frame correlation, which is updated by incorporating sequential information. The updated representation is used for depth estimation. Besides, we tackle VO as a self-supervised image generation task and take advantage of Generative Adversarial Networks (GAN). The generator learns to estimate depth and pose to generate a warped target image. The discriminator evaluates the quality of generated image with high-level structural perception that overcomes the problem of pixel-wise loss in previous methods. Experiments on KITTI and Cityscapes datasets show that our method obtains more accurate depth with details preserved and predicted pose outperforms state-of-theart self-supervised methods significantly.

show abstract

UnDEMoN: Unsupervised Deep Network for Depth and Ego-Motion Estimation

Cited by 23 publications

References 23 publications

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Constant Velocity Constraints for Self-Supervised Monocular Depth Estimation

Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry

Contact Info

Product

Resources

About