HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation

Abstract

Self-supervised learning shows great potential in monocular depth estimation, using image sequences as the only source of supervision. Although people try to use the high-resolution image for depth estimation, the accuracy of prediction has not been significantly improved. In this work, we find the core reason comes from the inaccurate depth estimation in large gradient regions, making the bilinear interpolation error gradually disappear as the resolution increases. To obtain more accurate depth estimation in large gradient regions, it is necessary to obtain high-resolution features with spatial and semantic information. Therefore, we present an improved DepthNet, HR-Depth, with two effective strategies: (1) re-design the skip-connection in DepthNet to get better high-resolution features and (2) propose feature fusion Squeeze-and-Excitation(fSE) module to fuse feature more efficiently. Using Resnet-18 as the encoder, HR-Depth surpasses all previous state-of-the-art(SoTA) methods with the least parameters at both high and low resolution. Moreover, previous SoTA methods are based on fairly complex and deep networks with a mass of parameters which limits their real applications. Thus we also construct a lightweight network which uses MobileNetV3 as encoder. Experiments show that the lightweight network can perform on par with many large models like Monodepth2 at high-resolution with only20%parameters. All codes and models will be available at https://github.com/shawLyu/HR-Depth.

MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer

“…Thus, to achieve good performance, this formulation requires the network to accurately perceive the scene structure: a challenging task, especially for regions with hard to distinguish foreground objects from the background. Current SoTA networks [57,30] rely on traditional convolutional layers for aggregating context information and gradually lift the receptive field of the network through a cascade of layers and strided convolution [40]. However, given the intrinsic locality of the convolution operator, CNNs hardly model long-range appearance similarity among objects, in particular within the shallowest features.…”

“…Considering the context difference between features at different scales, e.g. higher resolution features favour fine-grained details, we enhance cross-scale feature fusion with both spatial and channel attention mechanisms [30,69] (i.e., our Atten Block). Finally, four heads -made of two convolutional layers and a Sigmoid activation -are in charge of disparity (inverse depth) prediction from corresponding aggregated features, outputting maps at full, 1 2 , 1 4 , 1 8 resolution respectively.…”

“…Following [17,30,69,57], our PoseNet favors a simple, yet effective implementation. Specifically, our PoseNet uses the lightweight structure of ResNet18 [20].…”

“…Overall, network training requires about 15 hours. In our experiments, we adopt the same data augmentation detailed in [17,30].…”

“…Different kinds of backbone, such as VGGNet, ResNet, HRNet and PackNet, made their way into the self-supervised monocular depth estimation task [71,17,69,18]. Moreover, to improve the feature extraction and processing ability, new frameworks like HR-Depth [30] and CADepth [57] also introduced attention modules. However, we argue that a shared shortcoming of existing self-supervised models falls in the reduced receptive field of Convolutional Neural Networks (CNNs).…”