CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth

Fácil, José M.; Ummenhofer, Benjamin; Zhou, Huizhong; Montesano, Luis; Brox, Thomas; Civera, Javier

doi:10.1109/cvpr.2019.01210

Cited by 99 publications

(51 citation statements)

References 50 publications

(85 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, they did not consider strategies for optimal mixing, or study the impact of combining multiple datasets. Similarly, Facil et al [43] used multiple datasets with a naive mixing strategy for learning monocular depth with known camera intrinsics. Their test data is very similar to half of their training collection, namely RGB-D recordings of indoor scenes.…”

Section: Related Workmentioning

confidence: 99%

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

Ranftl

Lasinger²,

Hafner

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

902

603

View full text Add to dashboard Cite

The success of monocular depth estimation relies on large and diverse training sets. Due to the challenges associated with acquiring dense ground-truth depth across different environments at scale, a number of datasets with distinct characteristics and biases have emerged. We develop tools that enable mixing multiple datasets during training, even if their annotations are incompatible. In particular, we propose a robust training objective that is invariant to changes in depth range and scale, advocate the use of principled multi-objective learning to combine data from different sources, and highlight the importance of pretraining encoders on auxiliary tasks. Armed with these tools, we experiment with five diverse training datasets, including a new, massive data source: 3D films. To demonstrate the generalization power of our approach we use zero-shot cross-dataset transfer, i.e. we evaluate on datasets that were not seen during training. The experiments confirm that mixing data from complementary sources greatly improves monocular depth estimation. Our approach clearly outperforms competing methods across diverse datasets, setting a new state of the art for monocular depth estimation.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

Ranftl

Lasinger²,

Hafner

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

902

603

View full text Add to dashboard Cite

show abstract

“…[18][19][20][21]. Besides, recent developments have shown that the pixel-level depth map can be recovered from a single image in an end-to-end manner based on deep learning [22]. A variety of neural networks have manifested their effectiveness to address the monocular depth estimation, such as convolutional neural networks (CNNs) [23], recurrent neural networks (RNNs) [24], variational auto-encoders (VAEs) [25] and generative adversarial networks (GANs) [26].…”

Section: Introductionmentioning

confidence: 99%

Monocular depth estimation based on deep learning: An overview

Zhao¹,

Sun²,

Zhang³

et al. 2020

Sci. China Technol. Sci.

238

View full text Add to dashboard Cite

Depth information is important for autonomous systems to perceive environments and estimate their own state. Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints. Meanwhile, the predicted depth maps are sparse. Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy. Meanwhile, dense depth maps are estimated from single images by deep neural networks in an end-to-end manner. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed subsequently. Therefore, we survey the current monocular depth estimation methods based on deep learning in this review. Initially, we conclude several widely used datasets and evaluation indicators in deep learning-based depth estimation. Furthermore, we review some representative existing methods according to different training manners: supervised, unsupervised and semi-supervised. Finally, we discuss the challenges and provide some ideas for future researches in monocular depth estimation.

show abstract

“…Several recent works are proposed to tackle this practical open-world problem for stereo matching [62,50,49], but few work focus on online monocular depth adaptation. Compared with online stereo, online mono-depth learning is even more challenging from two aspects: (i) scale ambiguity and lack of support from geometrical information inherent to a monocular setting make the model extremely dependent on domain-specific visual features and prone to overfit on the current domain [11,10]; (ii) the environmental variation (e.g., speed or scene changes) introduces additional challenges both for depth or pose estimation, making the whole model fragile. On the top of these, as deep networks are not flexible in an online learning setting, there is catastrophic forgetting issue.…”

Section: Introductionmentioning

confidence: 99%

Online Depth Learning Against Forgetting in Monocular Videos

Zhang

Lathuilière

Ricci

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Online depth learning is the problem of consistently adapting a depth estimation model to handle a continuously changing environment. This problem is challenging due to the network easily overfits on the current environment and forgets its past experiences. To address such problem, this paper presents a novel Learning to Prevent Forgetting (LPF) method for online mono-depth adaptation to new target domains in unsupervised manner. Instead of updating the universal parameters, LPF learns adapter modules to efficiently adjust the feature representation and distribution without losing the pre-learned knowledge in online condition. Specifically, to adapt temporal-continuous depth patterns in videos, we introduce a novel meta-learning approach to learn adapter modules by combining online adaptation process into the learning objective. To further avoid overfitting, we propose a novel temporal-consistent regularization to harmonize the gradient descent procedure at each online learning step. Extensive evaluations on realworld datasets demonstrate that the proposed method, with very limited parameters, significantly improves the estimation quality.

show abstract

CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth

Cited by 99 publications

References 50 publications

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

Monocular depth estimation based on deep learning: An overview

Online Depth Learning Against Forgetting in Monocular Videos

Contact Info

Product

Resources

About