RigNet: Repetitive Image Guided Network for Depth Completion

Yan, Zhiqiang; Wang, Kun; Li, Xiang; Zhang, Zhenyu; Xu, Baobei; Li, Jun; Yang, Jian

doi:10.48550/arxiv.2107.13802

Cited by 6 publications

(24 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image-Guided methods [4], [8], [13], [14], [17]- [19], [31]- [34] utilize multi-modal information to facilitate dense depth completion. The multi-modal data includes RGB images, semantic images, and surface normal's, which act as reference images for generating dense depth maps.…”

Section: B Image-guided Methodsmentioning

confidence: 99%

“…The two branch network architectures [12]- [14], [17], [18] consist of RGB and sparse depth map branches. The RGB branch extracts color dominant information, e.g., object boundaries, which is actively fused with a sparse depth map branch at multiple stages.…”

Section: B Image-guided Methodsmentioning

confidence: 99%

“…PENET [13], utilized concatenation and addition operations to perform fusion at multiple levels. Recently, RigNet [14] proposed an efficient guidance algorithm to fuse and guide the sparse depth branch. In addition to RGB image and sparse depth map, DeepLiDAR [12] introduced learning pixel-wise surface normal's of the scene, which is fused through addition with RGB branch to generate dense depth maps.…”

Section: B Image-guided Methodsmentioning

confidence: 99%

“…These methods utilize convolutional neural networks and combine sparse Li-DAR data with different modalities, e.g. RGB images [13], [14], semantic maps [16], [21], [22], and surface normal's [12]. These modalities act as guidance and significantly help in the recovery of missing depth values in the sparse maps.…”

Section: Introductionmentioning

confidence: 99%

“…The combination of early and late fusion between the modalities enhances the depth completion performance. Recently, RigNet [14] proposed a repetitive architecture, which repeats Hourglass network architecture for color images at multiple levels to extract features and fuse them with depth generation branch to create structure-detailed dense depth maps. These methods rely heavily only on color images to extract color dominant information such as object boundaries to complete sparse maps.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

SemAttNet: Toward Attention-Based Semantic Aware Guided Depth Completion

et al. 2022

View full text Add to dashboard Cite

Depth completion involves recovering a dense depth map from a sparse map and an RGB image. Recent approaches focus on utilizing color images as guidance images to recover depth at invalid pixels. However, color images alone are not enough to provide the necessary semantic understanding of the scene. Consequently, the depth completion task suffers from sudden illumination changes in RGB images (e.g., shadows). In this paper, we propose a novel three-branch backbone comprising color-guided, semanticguided, and depth-guided branches. Specifically, the color-guided branch takes a sparse depth map and RGB image as an input and generates color depth which includes color cues (e.g., object boundaries) of the scene. The predicted dense depth map of color-guided branch along-with semantic image and sparse depth map is passed as input to semantic-guided branch for estimating semantic depth. The depth-guided branch takes sparse, color, and semantic depths to generate the dense depth map. The color depth, semantic depth, and guided depth are adaptively fused to produce the output of our proposed three-branch backbone. In addition, we also propose to apply semantic-aware multi-modal attention-based fusion block (SAMMAFB) to fuse features between all three branches. We further use CSPN++ with Atrous convolutions to refine the dense depth map produced by our three-branch backbone. Extensive experiments show that our model achieves state-of-the-art performance in the KITTI depth completion benchmark at the time of submission.INDEX TERMS State-of-the-art Depth Completion approach on KITTI depth completion benchmark, Attention-based fusion for depth completion, Semantic-guided depth completion

show abstract

Section: B Image-guided Methodsmentioning

confidence: 99%

Section: B Image-guided Methodsmentioning

confidence: 99%

Section: B Image-guided Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

SemAttNet: Toward Attention-Based Semantic Aware Guided Depth Completion

et al. 2022

View full text Add to dashboard Cite

show abstract

TbNet: three-branch network to guide deep completion

Hongkai¹,

Chi-Te²,

Ching-Hsiang³

et al. 2022

5th International Conference on Computer Information Science and Application Technology (CISAT 2022)

View full text Add to dashboard Cite

Sensing and Artificial Perception for Robots in Precision Forestry: A Survey

Ferreira,

Portugal,

Andrada

et al. 2023

Robotics

View full text Add to dashboard Cite

Artificial perception for robots operating in outdoor natural environments, including forest scenarios, has been the object of a substantial amount of research for decades. Regardless, this has proven to be one of the most difficult research areas in robotics and has yet to be robustly solved. This happens namely due to difficulties in dealing with environmental conditions (trees and relief, weather conditions, dust, smoke, etc.), the visual homogeneity of natural landscapes as opposed to the diversity of natural obstacles to be avoided, and the effect of vibrations or external forces such as wind, among other technical challenges. Consequently, we propose a new survey, describing the current state of the art in artificial perception and sensing for robots in precision forestry. Our goal is to provide a detailed literature review of the past few decades of active research in this field. With this review, we attempted to provide valuable insights into the current scientific outlook and identify necessary advancements in the area. We have found that the introduction of robotics in precision forestry imposes very significant scientific and technological problems in artificial sensing and perception, making this a particularly challenging field with an impact on economics, society, technology, and standards. Based on this analysis, we put forward a roadmap to address the outstanding challenges in its respective scientific and technological landscape, namely the lack of training data for perception models, open software frameworks, robust solutions for multi-robot teams, end-user involvement, use case scenarios, computational resource planning, management solutions to satisfy real-time operation constraints, and systematic field testing. We argue that following this roadmap will allow for robotics in precision forestry to fulfil its considerable potential.

show abstract

RigNet: Repetitive Image Guided Network for Depth Completion

Cited by 6 publications

References 61 publications

SemAttNet: Toward Attention-Based Semantic Aware Guided Depth Completion

SemAttNet: Toward Attention-Based Semantic Aware Guided Depth Completion

TbNet: three-branch network to guide deep completion

Sensing and Artificial Perception for Robots in Precision Forestry: A Survey

Contact Info

Product

Resources

About