Attention-based context aggregation network for monocular depth estimation

Chen, Yuru; Zhao, Haitao; Hu, Zhengwei; Peng, Jingchao

doi:10.1007/s13042-020-01251-y

Cited by 66 publications

(33 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We will further explore the possibilities of deploying proposed ideas on interesting applications such as image manipulation or editing, image-to-image transfer, 3D view rendering, contrast enhancement, refocusing, object recognition, realistic integration of virtual objects in augmented reality. [32] 0.586 DeepLabV3+ [33] 0.575 Multi-Task Light-Weight-RefineNet [34] 0.565 RelativeDepth [35] 0.538 SC-SfMLearner-ResNet18 [36] 0.536 SDC-Depth [37] 0.497 ACAN [38] 0.496 DORN [39] 0.509 SENet-154 [40] 0.530 Our (3DBGES-UNet) 0.1857…”

Section: Discussionmentioning

confidence: 99%

“…We choice to compare models not using extra training data. The comparison is performed with Zhu et al [31] (SOM), Xu et al [32] DeepLabV3+ [33], Multi-Task Light-Weight-RefineNet [34], Rel-ativeDepth [35], SC-SfMLearner-ResNet18 [36], SDC-Depth [37], ACAN [38], DORN [39], and SENet-154 [40] depth estimation methods. The results in Table 1 and Table 2 are quite impressive and onpar with state-of-the-art methods.…”

Section: Evaluation and Comparative Analysismentioning

confidence: 99%

See 1 more Smart Citation

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

Sharma

Tushar

et al. 2020

2020 International Conference on 3D Immersion (IC3D)

View full text Add to dashboard Cite

The task of predicting smooth and edge-consistent depth maps is notoriously difficult for single image depth estimation. This paper proposes a novel Bilateral Grid based 3D convolutional neural network, dubbed as 3DBG-UNet, that parameterizes high dimensional feature space by encoding compact 3D bilateral grids with UNets and infers sharp geometric layout of the scene. Further, another novel 3DBGES-UNet model is introduced that integrate 3DBG-UNet for inferring an accurate depth map given a single color view. The 3DBGES-UNet concatenates 3DBG-UNet geometry map with the inception network edge accentuation map and a spatial object's boundary map obtained by leveraging semantic segmentation and train the UNet model with ResNet backbone. Both models are designed with a particular attention to explicitly account for edges or minute details. Preserving sharp discontinuities at depth edges is critical for many applications such as realistic integration of virtual objects in AR video or occlusion-aware view synthesis for 3D display applications.The proposed depth prediction network achieves state-of-theart performance in both qualitative and quantitative evaluations on the challenging NYUv2-Depth data. The code and corresponding pre-trained weights will be made publicly available.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Evaluation and Comparative Analysismentioning

confidence: 99%

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

Sharma

Tushar

et al. 2020

2020 International Conference on 3D Immersion (IC3D)

View full text Add to dashboard Cite

show abstract

“…In recent years, deep convolutional networks have been applied to depth estimation and have achieved excellent results such as [2][3][4][5][6][7][8][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]. Now we generally considered that the beginning of the depth estimation of a single image based on deep learning is Eigen et al [2].…”

Section: A Monocular Depth Estimationmentioning

confidence: 99%

“…Laina et al [7] proposed a network based on FCRN (Fully Convolutional Residual Networks) to a depth map or depth maps. Yuru et al [24] added an attention mechanism to the classification algorithm, combined with contextual content, and it also used the soft classification method to improve the quality of prediction depth. Wu et al [23] applied (Atrous Spatial Pyramid Pooling) ASPP to depth estimation tasks.…”

Section: A Monocular Depth Estimationmentioning

confidence: 99%

Monocular Depth Estimation Based on Multi-Scale Depth Map Fusion

et al. 2021

View full text Add to dashboard Cite

Monocular depth estimation is a basic task in machine vision. In recent years, the performance of monocular depth estimation has been greatly improved. However, most depth estimation networks are based on a very deep network to extract features that lead to a large amount of information lost. The loss of object information is particularly serious in the encoding and decoding process. This information loss leads to the estimated depth maps lacking object structure detail and have non-clear edges. Especially in a complex indoor environment, which is our research focus in this paper, the consequences of this loss of information are particularly serious. To solve this problem, we propose a Dense feature fusion network that uses a feature pyramid to aggregate various scale features. Furthermore, to improve the fusion effectiveness of decoded object contour information and depth information, we propose an adaptive depth fusion module, which allows the fusion network to fuse various scale depth maps adaptively to increase object information in the predicted depth map. Unlike other work predicting depth maps relying on U-NET architecture, our depth map predicted by fusing multi-scale depth maps. These depth maps have their own characteristics. By fusing them, we can estimate depth maps that not only include accurate depth information but also have rich object contour and structure detail. Experiments indicate that the proposed model can predict depth maps with more object information than other prework, and our model also shows competitive accuracy. Furthermore, compared with other contemporary techniques, our method gets state-of-the-art in edge accuracy on the NYU Depth V2 dataset.

show abstract

“…To capture more context information, traditional CNN-based methods mainly refer to enlarging receptive fields of convolution operations, such as using large-size kernels, dilated convolution [ 9 ], ASPP [ 10 ] and DenseASPP [ 11 ], etc. Recently, benefitting from the nice capability to capture context information, the attention mechanism has been widely applied to monocular depth estimation [ 12 , 13 , 14 ]. Combining traditional CNN-based models with attention mechanism hugely advances the ability to capture more context information.…”

Section: Introductionmentioning

confidence: 99%

DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Lai

Tian

et al. 2021

Sensors

View full text Add to dashboard Cite

Pyramid architecture is a useful strategy to fuse multi-scale features in deep monocular depth estimation approaches. However, most pyramid networks fuse features only within the adjacent stages in a pyramid structure. To take full advantage of the pyramid structure, inspired by the success of DenseNet, this paper presents DCPNet, a densely connected pyramid network that fuses multi-scale features from multiple stages of the pyramid structure. DCPNet not only performs feature fusion between the adjacent stages, but also non-adjacent stages. To fuse these features, we design a simple and effective dense connection module (DCM). In addition, we offer a new consideration of the common upscale operation in our approach. We believe DCPNet offers a more efficient way to fuse features from multiple scales in a pyramid-like network. We perform extensive experiments using both outdoor and indoor benchmark datasets (i.e., the KITTI and the NYU Depth V2 datasets) and DCPNet achieves the state-of-the-art results.

show abstract

Attention-based context aggregation network for monocular depth estimation

Cited by 66 publications

References 40 publications

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

A Novel 3D-Unet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation

Monocular Depth Estimation Based on Multi-Scale Depth Map Fusion

DCPNet: A Densely Connected Pyramid Network for Monocular Depth Estimation

Contact Info

Product

Resources

About