Monocular 3D Object Detection via Feature Domain Adaptation

Ye, Xiaoqing; Liang, Dingkong; Shi, Yifeng; Li, Yingying; Tan, Xiao; Feng, Jianfeng; Ding, Errui; Wen, Shifeng

doi:10.1007/978-3-030-58545-7_2

Cited by 35 publications

(19 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The learning-based methods [43,6,36,37] directly regress the distance of objects by adding distance branches to 2D object detectors, which are simple and efficient. The pseudo-LiDAR-based methods [40,27,42] first predict the depth map of an input image using an external monocular depth estimator, then predict the distance of objects from the estimated depth map using a point-cloudbased 3D object detector. Though the explicit depth cues from the the estimated depth map can ease the distance prediction, the generalization of the methods is bounded by that of the monocular depth estimators [35].…”

Section: Monocular 3d Object Detectionmentioning

confidence: 99%

“…MonoRCNN directly predicts 3D bounding boxes from RGB images with a compact architecture, making the training and inference much more simple and efficient than those depth-based and video-based methods. Specifically, MonoRCNN runs 3 times faster than D 4 LCN [7], and 5 times faster than AM3D [27], DA-3Ddet [42], and PatchNet [26].…”

Section: Comparisons On the Kitti Benchmarkmentioning

confidence: 99%

“…The learning-based methods [43,6,36,37] directly learn a mapping from input images to the distance. The pseudo-LiDAR based meth-ods [40,27,42] first regress the depth map of an input image and then predict the distance of objects with the depth map. The 3D-anchor-based methods [1,2,7] break the distance prediction into the region proposal and the offset regression.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Geometry-based Distance Decomposition for Monocular 3D Object Detection

Shi

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Monocular 3D object detection is of great significance for autonomous driving but remains challenging. The core challenge is to predict the distance of objects in the absence of explicit depth information. Unlike regressing the distance as a single variable in most existing methods, we propose a novel geometry-based distance decomposition to recover the distance by its factors. The decomposition factors the distance of objects into the most representative and stable variables, i.e. the physical height and the projected visual height in the image plane. Moreover, the decomposition maintains the self-consistency between the two heights, leading to the robust distance prediction when both predicted heights are inaccurate. The decomposition also enables us to trace the cause of the distance uncertainty for different scenarios. Such decomposition makes the distance prediction interpretable, accurate, and robust. Our method directly predicts 3D bounding boxes from RGB images with a compact architecture, making the training and inference simple and efficient. The experimental results show that our method achieves the state-of-the-art performance on the monocular 3D Object Detection and Bird's Eye View tasks on the KITTI dataset, and can generalize to images with different camera intrinsics.

show abstract

Section: Monocular 3d Object Detectionmentioning

confidence: 99%

Section: Comparisons On the Kitti Benchmarkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Geometry-based Distance Decomposition for Monocular 3D Object Detection

Shi

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Following Frustum PointNet [33], most coordinate-based methods [43,46,28,44] Gradient boosting [13,14] is a general learning framework that combines multiple weak learners into a single strong one in an iterative fashion. Let L(x) be the risk of ensemble models, the algorithm devotes to seek an approximation h(x) to minimize L(y * , x) = Ψ(y * , h(x)), where y * is a target value, Ψ(•) is the loss function.…”

Section: Confidence-aware Localization Boostingmentioning

confidence: 99%

“…Typical coordinate-based methods [43,46,28,44] perform 3D detection based on 2D RoIs, which is similar to two-stage 2D detection frameworks, such as Faster R-CNN [35]. In a two-stage 2D object detection framework, the second stage reuses the features from the first stage via RoIPooling [35] or RoIAlign operators [19] guided by ROI proposals, and then a small decoder is used for localization refinement.…”

Section: Global Context Encodingmentioning

confidence: 99%

Progressive Coordinate Transforms for Monocular 3D Object Detection

Wang,

Zhang,

Zhu

et al. 2021

Preprint

View full text Add to dashboard Cite

Recognizing and localizing objects in the 3D space is a crucial ability for an AI agent to perceive its surrounding environment. While significant progress has been achieved with expensive LiDAR point clouds, it poses a great challenge for 3D object detection given only a monocular image. While there exist different alternatives for tackling this problem, it is found that they are either equipped with heavy networks to fuse RGB and depth information or empirically ineffective to process millions of pseudo-LiDAR points. With in-depth examination, we realize that these limitations are rooted in inaccurate object localization. In this paper, we propose a novel and lightweight approach, dubbed Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations. Specifically, a localization boosting mechanism with confidence-aware loss is introduced to progressively refine the localization prediction. In addition, semantic image representation is also exploited to compensate for the usage of patch proposals. Despite being lightweight and simple, our strategy leads to superior improvements on the KITTI and Waymo Open Dataset monocular 3D detection benchmarks. At the same time, our proposed PCT shows great generalization to most coordinatebased 3D detection frameworks. The code is available at: https://github.com/ amazon-research/progressive-coordinate-transforms.

show abstract

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection

Dai

Ding

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Leveraging LiDAR-based detectors or real LiDAR point data to guide monocular 3D detection has brought significant improvement, e.g., Pseudo-LiDAR methods. However, the existing methods usually apply non-end-to-end training strategies and insufficiently leverage the LiDAR information, where the rich potential of the LiDAR data has not been well exploited. In this paper, we propose the Cross-Modality Knowledge Distillation (CMKD) network for monocular 3D detection to efficiently and directly transfer the knowledge from LiDAR modality to image modality on both features and responses. Moreover, we further extend CMKD as a semi-supervised training framework by distilling knowledge from large-scale unlabeled data and significantly boost the performance. Until submission, CMKD ranks 1 st among the monocular 3D detectors with publications on both KITTI test set and Waymo val set with significant performance gains compared to previous state-of-the-art methods. Our code will be released at https://github.com/Cc-Hy/CMKD.

show abstract

Monocular 3D Object Detection via Feature Domain Adaptation

Cited by 35 publications

References 34 publications

Geometry-based Distance Decomposition for Monocular 3D Object Detection

Geometry-based Distance Decomposition for Monocular 3D Object Detection

Progressive Coordinate Transforms for Monocular 3D Object Detection

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection

Contact Info

Product

Resources

About