Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications

Feng, Xue; Zhuo, Guirong; Huang, Ziyuan; Fu, Wufei; Wu, Zhijun; Ang, Marcelo H.

doi:10.1109/iros45743.2020.9340802

Cited by 74 publications

(47 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 7 compares our method with other methods (J. Jiang et al., 2019; Liu et al., 2020; Xue et al. 2020), and our method has the lowest RMSE value. The REL value is slightly greater than the value in Liu et al.…”

Section: Experiments Of 3doimmentioning

confidence: 89%

Deep learning‐based object identification with instance segmentation and pseudo‐LiDAR point cloud for work zone safety

Shen

Yan

Xiong

2021

Computer aided Civil Eng

View full text Add to dashboard Cite

Automated object identification in three‐dimensional (3D) space is crucial for work zone safety, such as compliance with construction rules and preventing workplace injuries and deaths. However, it is greatly challenged by some factors like high‐quality detection, high‐quality instance segmentation, few engineering object datasets with masks, and accurate 3D object understanding due to scale variations and limited cues in the 3D world. Traditional hand‐crafted methods suffer from these challenges. Our key insight is to use 2D object detection, instance segmentation and camera vision to compute pseudo‐light detection and ranging (LiDAR) point cloud for 3D object identification. On the one hand, an enhanced feature pyramid network is proposed to extract more fine‐grained object features, and an improved cascade mask R‐CNN is applied to detect bounding boxes and masks for all 2D objects efficiently. Moreover, the AIM dataset for heavy equipment detection is augmented, and a new object class with the bounding box and mask is added. On the other hand, pseudo‐LiDAR point clouds of objects based on bounding boxes and masks are recovered from a monocular image by deep learning, automatic camera parameter estimation, vision‐based method, and space filter. Extensive experiments and analyses show that the new methodology can identify 3D objects and automatically analyze work zone safety. The proposed object detection model has achieved state‐of‐the‐art results on the AIM dataset and 97.2% in mean average precision for the augmented dataset. The collision detection model using pseudo‐LiDAR point cloud has obtained 95.99% in accuracy. The new model will serve as a baseline to support 3D object identification research for other 3D tasks.

show abstract

Section: Experiments Of 3doimmentioning

confidence: 89%

Deep learning‐based object identification with instance segmentation and pseudo‐LiDAR point cloud for work zone safety

Shen

Yan

Xiong

2021

Computer aided Civil Eng

View full text Add to dashboard Cite

show abstract

“…Hence, the depth and ego-motion estimates from these methods suffer from scale-inconsistency along with the global scale-ambiguity present in monocular vision. Therefore, ground truth LiDAR depth maps [4] or camera height [25] are used during inference to recover perimage scale.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation

Chawla¹,

Arnav²,

Zonooz³

2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Dense depth estimation is essential to sceneunderstanding for autonomous driving. However, recent selfsupervised approaches on monocular videos suffer from scaleinconsistency across long sequences. Utilizing data from the ubiquitously copresent global positioning systems (GPS), we tackle this challenge by proposing a dynamically-weighted GPS-to-Scale (g2s) loss to complement the appearance-based losses. We emphasize that the GPS is needed only during the multimodal training, and not at inference. The relative distance between frames captured through the GPS provides a scale signal that is independent of the camera setup and scene distribution, resulting in richer learned feature representations. Through extensive evaluation on multiple datasets, we demonstrate scale-consistent and -aware depth estimation during inference, improving the performance even when training with low-frequency GPS data.

show abstract

“…This unsatisfied performance may come from two reasons: the limited modeling power of the model and the limited supervision of loss functions. Many existing self-supervised depth estimation networks [16]- [18], [23], [24] adopt residual networks [25] based encoder-decoder architecture [26]. Such a depth network shows a powerful capacity for capturing local information and representing hierarchical abstract features, but is insufficient to distinguish important details from interfering information.…”

Section: Introductionmentioning

confidence: 99%

Visual Attention-based Self-supervised Absolute Depth Estimation using Geometric Priors in Autonomous Driving

Xiang¹,

Wang²,

An³

et al. 2022

Preprint

View full text Add to dashboard Cite

Although existing monocular depth estimation methods have made great progress, predicting an accurate absolute depth map from a single image is still challenging due to the limited modeling capacity of networks and the scale ambiguity issue. In this paper, we introduce a fully Visual Attention-based Depth (VADepth) network, where spatial attention and channel attention are applied to all stages. By continuously extracting the dependencies of features along the spatial and channel dimensions over a long distance, VADepth network can effectively preserve important details and suppress interfering features to better perceive the scene structure for more accurate depth estimates. In addition, we utilize geometric priors to form scale constraints for scale-aware model training. Specifically, we construct a novel scale-aware loss using the distance between the camera and a plane fitted by the ground points corresponding to the pixels of the rectangular area in the bottom middle of the image. Experimental results on the KITTI dataset show that this architecture achieves the state-of-theart performance and our method can directly output absolute depth without post-processing. Moreover, our experiments on the SeasonDepth dataset also demonstrate the robustness of our model to multiple unseen environments.

show abstract

Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications

Cited by 74 publications

References 23 publications

Deep learning‐based object identification with instance segmentation and pseudo‐LiDAR point cloud for work zone safety

Deep learning‐based object identification with instance segmentation and pseudo‐LiDAR point cloud for work zone safety

Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation

Visual Attention-based Self-supervised Absolute Depth Estimation using Geometric Priors in Autonomous Driving

Contact Info

Product

Resources

About