DTS-Depth: Real-Time Single-Image Depth Estimation Using Depth-to-Space Image Construction

Abstract

As most of the recent high-resolution depth-estimation algorithms are computationally so expensive that they cannot work in real time, the common solution is using a low-resolution input image to reduce the computational complexity. We propose a different approach, an efficient and real-time convolutional neural network-based depth-estimation algorithm using a single high-resolution image as the input. The proposed method efficiently constructs a high-resolution depth map using a small encoding architecture and eliminates the need for a decoder, which is typically used in the encoder–decoder architectures employed for depth estimation. The proposed algorithm adopts a modified MobileNetV2 architecture, which is a lightweight architecture, to estimate the depth information through the depth-to-space image construction, which is generally employed in image super-resolution. As a result, it realizes fast frame processing and can predict a high-accuracy depth in real time. We train and test our method on the challenging KITTI, Cityscapes, and NYUV2 depth datasets. The proposed method achieves low relative absolute error (0.028 for KITTI, 0.167 for CITYSCAPES, and 0.069 for NYUV2) while working at speed reaching 48 frames per second on a GPU and 20 frames per second on a CPU for high-resolution test images. We compare our method with the state-of-the-art methods on depth estimation, showing that our method outperforms those methods. However, the architecture is less complex and works in real time.

RT-ViT: Real-Time Monocular Depth Estimation Using Lightweight Vision Transformers

“…Lee et al [ 13 ] proposed a CNN-based method namely From big to small (BTS) which utilizes local planar guidance layers at different scales in the decoder stage that guides the feature maps to accurate depth predictions. We also provided challenging depth estimation results in previous research [ 14 , 15 ] in which we eliminate the complexity of the decoder in the encoder-decoder CNN architecture using depth-to-space (pixel-shuffle) image reconstruction. Although the previously stated methods attained relatively good results, the estimated depth in most of the stated methods has blurry results especially at the borders of the objects in the scene due to the inefficient encoding and decoding stages due to the local learning scheme naturally provided by the convolution algorithm.…”

“…Depth estimation is a critical task in a variety of computer vision applications, including 3D scene reconstruction from 2D images, medical 3D imaging, augmented reality, self-driving cars and robots, and 3D computer graphics and animations. The recent advances in depth estimation research have shown the effectiveness of the convolutional neural networks (CNNs) in performing such a task [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. The encoder-decoder CNN architectures are the most used architectures in the dense prediction tasks [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ] (image-like predictions such as semantic segmentation and depth estimation).…”

SD-Depth: Light-Weight Monocular Depth Estimation Using Space Depth CNN for Real-Time Applications

“…The DS module was employed in recent CNN-based depth methods [13][14] but the SD module was not presented as a down-sampling technique like we propose in this research. The suggested architecture outperforms state-of-the-art (SOTA) approaches for depth estimation, despite being simpler and less sophisticated than the SOTA methods.…”

Simplified digital content generation using single-shot depth estimation for full-color holographic printing system

“…Moreover, depth estimation is one of the effective methods that is utilized in several applications such as 3D imaging and scanning, background removal and separation, and 3D object rendering. Recently, depth estimation methods are proposed using effectiveness of modern convolutional neural networks (CNNs) [14,15].…”