DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume

Miao, Xiao-Rong; Bai, Y.; Duan, Haoran; Huang, Yawen; Wang, Fan; Xu, Xinxing; Long, Yang; Zheng, Yefeng

doi:10.1109/tcsvt.2023.3305776

Cited by 6 publications

(3 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, due to the modification of the loss function for learning structural errors, the feature fusion process gains significance. Instead of utilizing the feature network cited in [33], this approach adopts the feature fusion method referenced in [7,34,53]. Notably, selecting a smaller σ in the DoG measure reveals the structure of the target image.…”

Section: Depth Networkmentioning

confidence: 99%

“…In contrast, self-supervision has gained popularity due to its cost efficiency and independence from ground-truth data. A variety of methods have been introduced to address this challenge, including binocular consistency [3], visual synthesis [4], and the application of geometric constraints [5,6,7].…”

Section: Introductionmentioning

confidence: 99%

“…Then, Godard et al [10] integrated error algorithms, masks, networks, and training methods that became the basis for later methods. However, this view synthesis technique overlooks vital 3D geometric information, such as point clouds [11] and optical flows [7,12].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

MonoViT-3D: Self-Supervised Monocular Depth Estimation from 3D Geometric Constraints using Point Cloud Projection

Wen,

Tang,

Sun

et al. 2024

Preprint

View full text Add to dashboard Cite

Self-supervised monocular depth estimation leverages geometric constraints to create pseudolabels, eliminating the need for ground-truth data during training. In the field of 3D geometric constraints, the optical flow and iterative closest point methods are widely used to extract geometric constraints. However, the application of the existing 3D geometric constraint methods in self-supervised learning tasks increases network complexity by introducing additional submodules and iterative loops. In this paper, we propose a point cloud projection method that replaces the iterative closest point algorithm. Following depth prediction, our method utilizes the difference of 3D gaussian measure to extract 3D point cloud structures and the difference of 2D gaussian measure for the 2D structures derived from synthesized multiview surfaces. By projecting a 3D structure onto a 2D image, the errors between the target point cloud image and the synthesized views are estimated to enhance the training process of self-supervised learning. 3D geometric constraints form a point cloud optimization problem in our approach, requiring deep prediction networks with strong receptive fields for global features. Benefiting from the global receptive field of transformers, we use a vision transformer for self-supervised learning. Furthermore, we propose an encoder block to more effectively learn the patterns of 3D geometric constraints. Experiments show that we achieve strong performance on the KITTI dataset, achieving 0.095, 0.691, 4.243, and 0.171 Abs Rel, Sq Rel, RMSE, and RMSE log values for monocular depth estimation, respectively. Moreover, the model outperforms the comparison models in 3D generation tasks conducted on the Make3D dataset. All code is available at https://github.com/weentiaan/MonoViT-3D.

show abstract