Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark

Wang, Kun; Zhang, Zhenyu; Yan, Zhiqiang; Li, Xiang; Xu, Baobei; Li, Jun; Yang, Jian

doi:10.1109/iccv48922.2021.01575

Cited by 50 publications

(16 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, although combining Monodepth2 [18] and CycleGAN [17] could transform the nighttime images to 'fake' daytime images and reduce the domain shift between train and test images, the performances are also limited due to the loss of CycleGAN itself. ADFA [29] and RNW-Net [30], which is specialized to estimate depth of nighttime images, could reduce the domain shift between day and night images at the feature level, but the performance is limited by daytime results. ADDS-DepthNet [20] also use day-night image pair as input and could improve the depth estimation results of both the daytime and nighttime images to a certain extent, its performance can still be improved.…”

Section: Quantitative Resultsmentioning

confidence: 99%

“…[29] adapts a network trained on daytime images to work for nighttime images, aiming to transfer knowledge from daytime images to nighttime images. [30] propose Priors-Based Regularization to learn distribution knowledge from unpaired depth maps and Mapping-Consistent Image Enhancement module to enhance image visibility and contrast. [20] propose a domain-separated framework which partition the information of day-night image pairs into two complementary sub-spaces and relieve the influence of disturbing terms for all-day depth estimation.…”

Section: Nighttime Depth Estimationmentioning

confidence: 99%

See 1 more Smart Citation

GlocalFuse-Depth: Fusing Transformers and CNNs for All-day Self-supervised Monocular Depth Estimation

Zhang¹,

Chan²,

Wong³

2023

Preprint

View full text Add to dashboard Cite

In recent years, self-supervised monocular depth estimation has drawn much attention since it frees of depth annotations and achieved remarkable results on standard benchmarks. However, most of existing methods only focus on either daytime or nighttime images, thus their performance degrades on the other domain because of the large domain shift between daytime and nighttime images. To address this problem, in this paper we propose a two-branch network named GlocalFuse-Depth for self-supervised depth estimation of all-day images. The daytime and nighttime image in input image pair are fed into the two branches: CNN branch and Transformer branch, respectively, where both fine-grained details and global dependency can be efficiently captured. Besides, a novel fusion module is proposed to fuse multi-dimensional features from the two branches. Extensive experiments demonstrate that GlocalFuse-Depth achieves state-of-the-art results for all-day images on the Oxford RobotCar dataset, which proves the superiority of our method.

show abstract

Section: Quantitative Resultsmentioning

confidence: 99%

Section: Nighttime Depth Estimationmentioning

confidence: 99%

GlocalFuse-Depth: Fusing Transformers and CNNs for All-day Self-supervised Monocular Depth Estimation

Zhang¹,

Chan²,

Wong³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition, depth estimation also needs to consider some special scenarios, such as dark, foggy, rainy, and snowy weather. These scenes are very challenging scenes, and some recent studies have focused on these problems, such as Wang, K et al [63] research on depth estimation in night environments. With the widespread application of depth estimation, new research needs to consider special scenarios to make depth estimation more general.…”

Section: Discussionmentioning

confidence: 99%

Multi-Sensor Fusion Self-Supervised Deep Odometry and Depth Estimation

et al. 2022

View full text Add to dashboard Cite

This paper presents a new deep visual-inertial odometry and depth estimation framework for improving the accuracy of depth estimation and ego-motion from image sequences and inertial measurement unit (IMU) raw data. The proposed framework predicts ego-motion and depth with absolute scale in a self-supervised manner. We first capture dense features and solve the pose by deep visual odometry (DVO), and then combine the pose estimation pipeline with deep inertial odometry (DIO) by the extended Kalman filter (EKF) method to produce the sparse depth and pose with absolute scale. We then join deep visual-inertial odometry (DeepVIO) with depth estimation by using sparse depth and the pose from DeepVIO pipeline to align the scale of the depth prediction with the triangulated point cloud and reduce image reconstruction error. Specifically, we use the strengths of learning-based visual-inertial odometry (VIO) and depth estimation to build an end-to-end self-supervised learning architecture. We evaluated the new framework on the KITTI datasets and compared it to the previous techniques. We show that our approach improves results for ego-motion estimation and achieves comparable results for depth estimation, especially in the detail area.

show abstract

“…Watson et al [26] drew on the cost-volume method in multi-view depth estimation and propose a novel consistency loss to handle moving objects. Wang et al [27] proposed a statistical-based masking strategy, which uses dynamic statistics to adjust the number of pixels removed in the low-texture area.…”

Section: Related Workmentioning

confidence: 99%

Semantically guided self‐supervised monocular depth estimation

Sun

Wang

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

Depth information plays an important role in the vision‐related activities of robots and autonomous vehicles. An effective method to obtain 3D scene information is self‐supervised monocular depth estimation, which utilizes large and diverse monocular video datasets during the training process without the need for ground‐truth data. A novel multi‐task learning strategy that uses semantic information to guide the monocular depth estimation method while maintaining self‐supervision is proposed. An improved differential direct visual odometer (DDVO) combined with Pose‐Net is applied for achieving better pose prediction. Minimum reprojection loss with auto‐masking and semantic masking is used to remove the effects of low‐texture areas and moving dynamic‐class objects within scenes. Concurrently, the semantic masking is introduced into the DDVO pose predictor to filter moving objects and reduce the matching error between monocular sequence frames. In addition, PackNet is employed as the backbone of multi‐task learning to further improve the accuracy of deep prediction. The proposed method produces state‐of‐the‐art results for monocular depth estimation on the KITTI Eigen split benchmark, even outperforming supervised methods that have been trained using ground‐truth depth.

show abstract

Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark

Cited by 50 publications

References 39 publications

GlocalFuse-Depth: Fusing Transformers and CNNs for All-day Self-supervised Monocular Depth Estimation

GlocalFuse-Depth: Fusing Transformers and CNNs for All-day Self-supervised Monocular Depth Estimation

Multi-Sensor Fusion Self-Supervised Deep Odometry and Depth Estimation

Semantically guided self‐supervised monocular depth estimation

Contact Info

Product

Resources

About