Pseudo-Mono for Monocular 3D Object Detection in Autonomous Driving

Tao, Chongben; Cao, JieCheng; Wang, Chen; Zhang, Zufeng; Gao, Zhen

doi:10.1109/tcsvt.2023.3237579

Cited by 18 publications

(2 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The limitations mentioned above hinder practical applications to construction scenes. To overcome them, monocular 3D object detection from a single camera sensor has attracted more and more research attention due to its lower cost, higher frame rate and resolution, and broader deployment (Tao et al., 2023), which is more scalable and compelling. Currently, deep learning and 3D object detection data sets are rapidly evolving (Caesar et al., 2020; Geiger et al., 2012), and monocular 3D detection has made remarkable progress (Kim & Hwang, 2021; Shen et al., 2021; Shen et al., 2023).…”

Section: Introductionmentioning

confidence: 99%

Monocular 3D object detection for construction scene analysis

Shen,

Jiao,

Zhang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

Three‐dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high‐quality feature representations for depth estimation, and limited 3D cues from a single red‐green‐blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade‐based network with a transformer backbone and a boundary‐patch‐refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self‐supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo‐LiDAR point cloud method and a 3D object detection method with a density‐based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

Monocular 3D object detection for construction scene analysis

Shen,

Jiao,

Zhang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

show abstract

“…Automatic driving vehicles require the accurate detection of surrounding objects to enable appropriate planning from subsequent decision-making algorithms. In recent years, deep-learning-based object detection algorithms [1][2][3][4][5][6][7][8][9] have achieved remarkable results, and many publicly available datasets have been used to evaluate algorithm performance. These models often rely on supervised training on large-scale datasets with ground truth annotations.…”

Section: Introductionmentioning

confidence: 99%

An Object Detection Method Based on Feature Uncertainty Domain Adaptation for Autonomous Driving

Zhu

et al. 2023

Applied Sciences

Self Cite

View full text Add to dashboard Cite

The environment perception algorithm in autonomous driving is trained in the source domain, leading to domain drift and reduced detection accuracy in the target domain due to shifts in background feature distribution. To address this issue, a domain adaptive object detection algorithm based on feature uncertainty is proposed, which can improve the detection performance of object detection algorithms in unlabeled data. Firstly, a local alignment module based on channel information is proposed, which can obtain the model’s uncertainty about different domain data based on the feature channels obtained through the feature extraction network, achieving adaptive dynamic local alignment. Secondly, an instance-level alignment module guided by local feature uncertainty is proposed, which can obtain the corresponding instance-level uncertainty through ROI mapping. To improve the domain invariance of bounding box regression, a multi-class, multi-regression instance-level uncertainty alignment module is proposed, which can achieve spatial decoupling of classification and regression tasks, further improving the model’s domain adaptive ability. Finally, the effectiveness of the proposed algorithm is validated on Cityscapes, KITTI, and real vehicle data.

show abstract