Unsupervised Monocular Depth Estimation Based on Dual Attention Mechanism and Depth-Aware Loss

Ye, Xinchen; Zhang, Mingliang; Xu, Rui; Zhong, Wei; Fan, Xin; Li, Zhu; Zhang, Jiaao

doi:10.1109/icme.2019.00037

Cited by 5 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the idea of the image scale pyramid, Poggi et al [31] designed PyD-Net for real-time use. In recent years, Ye et al [32] introduced a dual-attention module to enhance feature representations. Attention mechanism has also been researched to improve the edges of depth maps [33] and general contextual information [34].…”

Section: Monocular Depth Estimationmentioning

confidence: 99%

SFA-MDEN: Semantic-Feature-Aided Monocular Depth Estimation Network Using Dual Branches

Wang

Zou

Wen

2021

Sensors

View full text Add to dashboard Cite

Monocular depth estimation based on unsupervised learning has attracted great attention due to the rising demand for lightweight monocular vision sensors. Inspired by multi-task learning, semantic information has been used to improve the monocular depth estimation models. However, multi-task learning is still limited by multi-type annotations. As far as we know, there are scarcely any large public datasets that provide all the necessary information. Therefore, we propose a novel network architecture Semantic-Feature-Aided Monocular Depth Estimation Network (SFA-MDEN) to extract multi-resolution depth features and semantic features, which are merged and fed into the decoder, with the goal of predicting depth with the support of semantics. Instead of using loss functions to relate the semantics and depth, the fusion of feature maps for semantics and depth is employed to predict the monocular depth. Therefore, two accessible datasets with similar topics for depth estimation and semantic segmentation can meet the requirements of SFA-MDEN for training sets. We explored the performance of the proposed SFA-MDEN with experiments on different datasets, including KITTI, Make3D, and our own dataset BHDE-v1. The experimental results demonstrate that SFA-MDEN achieves competitive accuracy and generalization capacity compared to state-of-the-art methods.

show abstract

Section: Monocular Depth Estimationmentioning

confidence: 99%

SFA-MDEN: Semantic-Feature-Aided Monocular Depth Estimation Network Using Dual Branches

Wang

Zou

Wen

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…MDP methods can predict the depth map using only one RGB image; however, the object boundaries in the generated depth map are not clear. To improve the sharpness at the boundaries of objects in the depth map, many researchers have begun to leverage the attention mechanism [10][11][12]. However, existing attention modules tend to focus on firstorder coarse information, which may fail to obtain welldistinguished features.…”

Section: Introductionmentioning

confidence: 99%

A multiscale dilated convolution and mixed-order attention-based deep neural network for monocular depth prediction

2022

SN Appl. Sci.

View full text Add to dashboard Cite

Recovering precise depth information from different scenes has become a popular subject in the semantic segmentation and virtual reality fields. This study presents a multiscale dilated convolution and mixed-order attention-based deep neural network for monocular depth recovery. Specifically, we design a multilevel feature enhancement scheme to enhance and fuse high-resolution and low-resolution features on the basis of mixed-order attention. Moreover, a multiscale dilated convolution module that combines four different dilated convolutions is explored for deriving multiscale information and increasing the receptive field. Recent studies have shown that the design of loss terms is crucial to depth prediction. Therefore, an efficient loss function that combines the ℓ1 loss, gradient loss, and classification loss is also designed to promote rich details. Experiments on three public datasets show that the presented approach achieves better performance than state-of-the-art depth prediction methods.

show abstract

“…Deep-learning-based methods have been widely used to estimate depth from images. Most of these methods are categorized into two classes, namely (1) depth prediction from a single image [11][12][13][14] and (2) binocular stereo matching [15][16][17][18]. Depth prediction from a single image generates a depth map for a single-view image, whereas binocular stereo matching takes as input a rectified image pair and outputs its disparity map.…”

Section: Introductionmentioning

confidence: 99%

A Unified Framework for Depth Prediction from a Single Image and Binocular Stereo Matching

et al. 2020

View full text Add to dashboard Cite

Depth information has long been an important issue in computer vision. The methods for this can be categorized into (1) depth prediction from a single image and (2) binocular stereo matching. However, these two methods are generally regarded as separate tasks, which are accomplished in different network architectures when using deep learning-based methods. This study argues that these two tasks can be achieved using only one network with the same weights. We modify existing networks for stereo matching to perform the two tasks. We first enable the network capable of accepting both a single image and an image pair by duplicating the left image when the right image is absent. Then, we introduce a training procedure that alternatively selects training samples of depth prediction from a single image and binocular stereo matching. In this manner, the trained network can perform both tasks and single-image depth prediction even benefits from stereo matching to achieve better performance. Experimental results on KITTI raw dataset show that our model achieves state-of-the-art performances for accomplishing depth prediction from a single image and binocular stereo matching in the same architecture.

show abstract

Unsupervised Monocular Depth Estimation Based on Dual Attention Mechanism and Depth-Aware Loss

Cited by 5 publications

References 14 publications

SFA-MDEN: Semantic-Feature-Aided Monocular Depth Estimation Network Using Dual Branches

SFA-MDEN: Semantic-Feature-Aided Monocular Depth Estimation Network Using Dual Branches

A multiscale dilated convolution and mixed-order attention-based deep neural network for monocular depth prediction

A Unified Framework for Depth Prediction from a Single Image and Binocular Stereo Matching

Contact Info

Product

Resources

About