Self‐supervised depth completion with multi‐view geometric constraints

Abstract

Self‐supervised learning‐based depth completion is a cost‐effective way for 3D environment perception. However, it is also a challenging task because sparse depth may deactivate neural networks. In this paper, a novel Sparse‐Dense Depth Consistency Loss (SDDCL) is proposed to penalize not only the estimated depth map with sparse input points but also consecutive completed dense depth maps. Combined with the pose consistency loss, a new self‐supervised learning scheme is developed, using multi‐view geometric constraints, to achieve more accurate depth completion results. Moreover, to tackle the sparsity issue of input depth, a Quasi Dense Representations (QDR) module with triplet branches for spatial pyramid pooling is proposed to produce more dense feature maps. Extensive experimental results on VOID, NYUv2, and KITTI datasets show that the method outperforms state‐of‐the‐art self‐supervised depth completion methods.

MonoViT-3D: Self-Supervised Monocular Depth Estimation from 3D Geometric Constraints using Point Cloud Projection

“…In contrast, self-supervision has gained popularity due to its cost efficiency and independence from ground-truth data. A variety of methods have been introduced to address this challenge, including binocular consistency [3], visual synthesis [4], and the application of geometric constraints [5,6,7].…”