Bilateral Cyclic Constraint and Adaptive Regularization for Unsupervised Monocular Depth Prediction

Abstract

Supervised learning methods to infer (hypothesize) depth of a scene from a single image require costly per-pixel ground-truth. We follow a geometric approach that exploits abundant stereo imagery to learn a model to hypothesize scene structure without direct supervision. Although we train a network with stereo pairs, we only require a single image at test time to hypothesize disparity or depth. We propose a novel objective function that exploits the bilateral cyclic relationship between the left and right disparities and we introduce an adaptive regularization scheme that allows the network to handle both the co-visible and occluded regions in a stereo pair. This process ultimately produces a model to generate hypotheses for the 3-dimensional structure of the scene as viewed in a single image. When used to generate a single (most probable) estimate of depth, our method outperforms state-of-the-art unsupervised monocular depth prediction methods on the KITTI benchmarks. We show that our method generalizes well by applying our models trained on KITTI to the Make3d dataset. Code and pre-trained models available at: https

CoMoDA: Continuous Monocular Depth Adaptation Using Past Experiences

“…Godard et al [12] further improved these results by adding left-right consistency and SSIM [39] terms to the image reconstruction loss. Notable improvements were achieved by enforcing left-right consistency between intermediate feature maps [46], cycle-consistency [29,40], or using temporal cues [22,46]. Some other self-supervised methods [30,1] utilize GANs [14] to improve image synthesis, thus also improving the predicted depth quality.…”

An Adaptive Framework for Learning Unsupervised Depth Completion

“…However, this weighting scheme is still static with respect to a given image. [10], [9], [32] proposed adaptive regularization in the spatial domain and over the course of optimization based on the local residual. However, their method considers only a single frame.…”

“…Hence, we have chosen an adaptive method [32] from the single image depth prediction literature. We provide a comparative study on VOID by applying the weighting scheme of [32] to [16] and VGG11 [31] in Table VI. Overall, [16] with [32] performs worse than using [16] + our γ alone.…”

Underwater Depth Estimation for Spherical Images

“…e study in [35] investigated the multimodality depth completion task with a self-supervised method by constructing a loss function with photometric constraints, and their method achieved the state of the art (SOTA) on the KITTI depth completion benchmark. e study in [36] exploited the bilateral cyclic relationship between stereo disparities and proposed an adaptive regularization scheme to handle covisible and occluded problems in a stereo pair.…”