Feature-Metric Loss for Self-supervised Learning of Depth and Egomotion

Shu, Chang; Yu, Kun; Duan, Zhixiang; Yang, Kuiyuan

doi:10.1007/978-3-030-58529-7_34

Cited by 184 publications

(142 citation statements)

References 47 publications

Supporting

Mentioning

141

Contrasting

Unclassified

Order By: Relevance

“…multi-scale prediction to prevent the training target from being trapped in the local minimum with gradient locality by bilinear sampling. Recent approaches add loss [33], networks such as an optical flow network for motion information supplementation [34,35], and a feature-metric network for semantic information addition [36] and reduce the performance difference between monocular and stereo-based depth estimation. However, this unsupervised learned depth is not guaranteed by a metric measure.…”

Section: Self-supervised Trainingmentioning

confidence: 99%

“…Since the depth discontinuity depends on the gradients δI t of the image, the edgeaware term is used together as in previous studies [17,36,37] to limit the high depth gradient δD t for the texture-less region.…”

Section: Depth Smoothness Lossmentioning

confidence: 99%

“…As mentioned above, recent research studies use a method of adding a network reinforcing feature or segmentation information [36,40] and a loss model for geometry or light [16,33]. Intuitively, feature and semantic information are not appropriate for depth prediction due to the characteristics of colonoscopy images.…”

Section: Improved Self-supervised Trainingmentioning

confidence: 99%

See 2 more Smart Citations

Unsupervised Monocular Depth Estimation for Colonoscope System Using Feedback Network

Hwang

Park

Kim

et al. 2021

Sensors

View full text Add to dashboard Cite

A colonoscopy is a medical examination used to check disease or abnormalities in the large intestine. If necessary, polyps or adenomas would be removed through the scope during a colonoscopy. Colorectal cancer can be prevented through this. However, the polyp detection rate differs depending on the condition and skill level of the endoscopist. Even some endoscopists have a 90% chance of missing an adenoma. Artificial intelligence and robot technologies for colonoscopy are being studied to compensate for these problems. In this study, we propose a self-supervised monocular depth estimation using spatiotemporal consistency in the colon environment. It is our contribution to propose a loss function for reconstruction errors between adjacent predicted depths and a depth feedback network that uses predicted depth information of the previous frame to predict the depth of the next frame. We performed quantitative and qualitative evaluation of our approach, and the proposed FBNet (depth FeedBack Network) outperformed state-of-the-art results for unsupervised depth estimation on the UCL datasets.

show abstract

Section: Self-supervised Trainingmentioning

confidence: 99%

Section: Depth Smoothness Lossmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Monocular Depth Estimation for Colonoscope System Using Feedback Network

Hwang

Park

Kim

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…With advances in deep networks, Eigen et al [2] showed that monocular depth estimation with deep neural network yields significant gains in accuracy and speed compared to the traditional attempts with handcrafted features. Starting with the work of Eigen et al [2], many learning-based methods [1,3,4,5,6,7,8,9,10,11] have been studied and showed high accuracy in overall depth evaluation metrics. However, they still produce noisy and temporally flickering depth maps because they perform depth estimation on each frame independently or do not use temporal information correctly.…”

Section: Introductionmentioning

confidence: 99%

“…Several video-based depth estimation methods have been studied, and the core idea behind these methods is to utilize temporal information. Temporal consistency can be explicitly constrained [1,7,8,9,12,13] or implicitly constrained using the recurrent neural networks [7,10,11]. Eom et al [7] proposed the recurrent model with a flow-guided memory unit for consistent depth.…”

Section: Introductionmentioning

confidence: 99%

Stad: Stable Video Depth Estimation

Lee¹,

Park

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

We present a method for estimating temporally stable depth video from a sequence of images. We extend the prior work aimed at video depth estimation, Neural-RGBD [1], which proposed to use temporal information by accumulating a depth probability volume over time. We propose three simple yet effective ideas to gain improvement: (1) temporal attention module to select and propagate only the meaningful temporal information, (2) geometric warping operation to warp neighbor features in the way of preserving geometry cues, and (3) scale-invariant loss to relieve the inherent scale ambiguity problem in monocular depth estimation task. We demonstrate the efficiency of proposed ideas by comparing our proposed network STAD with the state-of-the-arts. Moreover, we compare STAD with its per-frame network STAD-frame to show the importance of utilizing temporal information. The experimental results show that STAD significantly improved the baseline accuracy without a large parameter increase.

show abstract