Semantically-Guided Representation Learning for Self-Supervised Monocular Depth

Guizilini, Vitor; Hou, Rui; Li, Jie; Ambruş, Rareş; Gaidon, Adrien

doi:10.48550/arxiv.2002.12319

Cited by 22 publications

(37 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It demonstrates that when the fine-tuning dataset is similar to the test dataset, the finetuning method performs better than the data transformation method. The best domain adaptation method [138] has superior performance to the best unsupervised method [119]. Regarding the best semi-supervised method [119] and the best domain adaptation method [138], [119] outperforms [138].…”

Section: A Accuracymentioning

confidence: 95%

“…The best domain adaptation method [138] has superior performance to the best unsupervised method [119]. Regarding the best semi-supervised method [119] and the best domain adaptation method [138], [119] outperforms [138]. In particular, their accuracy metrics are almost equal, while the error metrics especially Sq Rel, RMSE and RMSE log are highly variable.…”

Section: A Accuracymentioning

confidence: 96%

“…Unlike previous work that computes the surface normals from the depth values first and then impose regularity, they directly regularize the depth values via the scale-invariant constraint. Guizilini et al [119] utilize semantic information to guide the geometric representation learning of MDE. The designed architecture is built within an unsupervised scheme [120].…”

Section: B Depth Estimation With Unsupervised Learningmentioning

confidence: 99%

See 2 more Smart Citations

Towards Real-Time Monocular Depth Estimation for Robotics: A Survey

Dong¹,

Garratt²,

Anavatti³

et al. 2021

Preprint

View full text Add to dashboard Cite

As an essential component for many autonomous driving and robotic activities such as ego-motion estimation, obstacle avoidance and scene understanding, monocular depth estimation (MDE) has attracted great attention from the computer vision and robotics communities. Over the past decades, a large number of methods have been developed. To the best of our knowledge, however, there is not a comprehensive survey of MDE. This paper aims to bridge this gap by reviewing 197 relevant articles published between 1970 and 2021. In particular, we provide a comprehensive survey of MDE covering various methods, introduce the popular performance evaluation metrics and summarize publically available datasets. We also summarize available open-source implementations of some representative methods and compare their performances. Furthermore, we review the application of MDE in some important robotic tasks. Finally, we conclude this paper by presenting some promising directions for future research. This survey is expected to assist readers to navigate this research field.

show abstract

Section: A Accuracymentioning

confidence: 95%

Section: A Accuracymentioning

confidence: 96%

Section: B Depth Estimation With Unsupervised Learningmentioning

confidence: 99%

See 1 more Smart Citation

Towards Real-Time Monocular Depth Estimation for Robotics: A Survey

Dong¹,

Garratt²,

Anavatti³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our weakly-supervised method achieves competitive subjective results with fully-supervised ones. [49], [50], [51], [47], [46], [54], semi-supervised [55], [56], [54], [9], [26], [10] and fully-supervised ones [1], [19], [2], [6], [3], [21], [4], [5], [22], [23], [14], [44], [45], [24] on two benchmark datasets, i.e., KITTI [42] and NYU Depth-v2 [52] datasets. Since the fully-supervised methods exploit the groundtruth HR depth maps for training, in theory it should work better than other learning manners.…”

Section: Performance Comparisonmentioning

confidence: 99%

Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data

Xu,

Bai,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Depth estimation from a single image is an active research topic in computer vision. The most accurate approaches are based on fully supervised learning models, which rely on a large amount of dense and high-resolution (HR) ground-truth depth maps. However, in practice, color images are usually captured with much higher resolution than depth maps, leading to the resolution-mismatched effect. In this paper, we propose a novel weakly-supervised framework to train a monocular depth estimation network to generate HR depth maps with resolution-mismatched supervision, i.e., the inputs are HR color images and the ground-truth are low-resolution (LR) depth maps. The proposed weakly supervised framework is composed of a sharing weight monocular depth estimation network and a depth reconstruction network for distillation. Specifically, for the monocular depth estimation network the input color image is first downsampled to obtain its LR version with the same resolution as the ground-truth depth. Then, both HR and LR color images are fed into the proposed monocular depth estimation network to obtain the corresponding estimated depth maps. We introduce three losses to train the network: 1) reconstruction loss between the estimated LR depth and the ground-truth LR depth; 2) reconstruction loss between the downsampled estimated HR depth and the ground-truth LR depth; 3) consistency loss between the estimated LR depth and the downsampled estimated HR depth. In addition, we design a depth reconstruction network from depth to depth. Through distillation loss, features between two networks maintain the structural consistency in affinity space, and finally the depth reconstruction network improve the monocular depth estimation network performance. Experimental results demonstrate that our method achieves superior performance than unsupervised and semi-supervised learning based schemes, and is competitive or even better compared to supervised ones.

show abstract

“…Guizilini et al [13] learned to compress and decompress detail-preserving representations by symmetrical packing and unpacking blocks. Other published methods were based upon edge and normal [48,49], Competitive Collaboration [36], semantic segmentation [14,22] and feature representations learning [41]. A state-of-theart framework was Monodepth2 proposed by Godard et al [11], which introduced a minimum re-projection loss to deal with occlusions and auto-masking scheme removing invalid pixels robustly.…”

Section: Self-supervised Monocular Depth Estimationmentioning

confidence: 99%

Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation

Yan¹,

Zhao²,

Bu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Self-supervised learning has shown very promising results for monocular depth estimation. Scene structure and local details both are significant clues for high-quality depth estimation. Recent works suffer from the lack of explicit modeling of scene structure and proper handling of details information, which leads to a performance bottleneck and blurry artefacts in predicted results. In this paper, we propose the Channel-wise Attention-based Depth Estimation Network (CADepth-Net) with two effective contributions: 1) The structure perception module employs the selfattention mechanism to capture long-range dependencies and aggregates discriminative features in channel dimensions, explicitly enhances the perception of scene structure, obtains the better scene understanding and rich feature representation. 2) The detail emphasis module re-calibrates channel-wise feature maps and selectively emphasizes the informative features, aiming to highlight crucial local details information and fuse different level features more efficiently, resulting in more precise and sharper depth prediction. Furthermore, the extensive experiments validate the effectiveness of our method and show that our model achieves the state-of-the-art results on the KITTI benchmark and Make3D datasets.

show abstract

Semantically-Guided Representation Learning for Self-Supervised Monocular Depth

Cited by 22 publications

References 33 publications

Towards Real-Time Monocular Depth Estimation for Robotics: A Survey

Towards Real-Time Monocular Depth Estimation for Robotics: A Survey

Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data

Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation

Contact Info

Product

Resources

About