MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation

Chen, Hansheng; Huang, Yuyao; Tian, Wei; Gao, Zhong-Ke; Liu, Xiong

doi:10.1109/cvpr46437.2021.01024

Cited by 94 publications

(54 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BB8 [37] and RTM3D [28] locate the corners of the 3D bounding box as keypoints, while PVNet [36] defines the keypoints by farthest point sampling and Deep MANTA [11] by handcrafted templates. On the other hand, dense correspondence methods [13,29,35,46,52] predict pixel-wise 3D coordinates within a cropped 2D region. Most existing geometry-based methods follow a two-stage strategy, where the intermediate representations (i.e., 2D-3D correspondences) are learned with a surrogate loss function, which is sub-optimal compared to end-to-end learning.…”

Section: Related Workmentioning

confidence: 99%

“…This inspired works such as DSAC [4], a smooth RANSAC with a finite hypothesis pool. Meanwhile, simple parametric distributions (e.g., normal distribution) are often used in predicting continuous variables [13,18,22,25,26,51], and mixture distributions can be employed to further capture ambiguity [3,5,31], e.g., ambiguous 6DoF pose [7]. In this paper, we propose yet a unique contribution: backpropagating a complicated continuous distribution derived from a nested optimization layer (the PnP layer), essentially making the continuous counterpart of Softmax tractable.…”

Section: End-to-end Correspondence Learningmentioning

confidence: 99%

“…( 5) are concerned with the reprojection errors at target and predicted pose respectively. The former is often used as a surrogate loss in previous work [6,12,13]. However, the first term alone cannot handle learning all 2D-3D points without imposing strict regularization, as the minimization could simply drive all the points to a concentrated location without pose discrimination.…”

Section: Overviewmentioning

confidence: 99%

“…Comparison to Implicit Differentiation Method Existing work on end-to-end PnP [9,12] derives a single solution of a particular solver y * = PnP (X) via implicit function theorem [19]. In the probabilistic framework, this is essentially the Laplace method that approximates the posterior by N (y * , Σ y * ), where both y * and Σ y * can be estimated by the PnP solver with analytical derivatives [13]. A special case is that, with Σ y * simplified to be homogeneous, the approximated KL divergence can be simplified to the L2 loss y * − y gt 2 used in [9].…”

Section: Overviewmentioning

confidence: 99%

See 3 more Smart Citations

EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

Chen¹,

Wang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Locating 3D objects from a single RGB image via Perspective-n-Points (PnP) is a long-standing problem in computer vision. Driven by end-to-end deep learning, recent studies suggest interpreting PnP as a differentiable layer, so that 2D-3D point correspondences can be partly learned by backpropagating the gradient w.r.t. object pose. Yet, learning the entire set of unrestricted 2D-3D points from scratch fails to converge with existing approaches, since the deterministic pose is inherently non-differentiable. In this paper, we propose the EPro-PnP, a probabilistic PnP layer for general end-to-end pose estimation, which outputs a distribution of pose on the SE(3) manifold, essentially bringing categorical Softmax to the continuous domain. The 2D-3D coordinates and corresponding weights are treated as intermediate variables learned by minimizing the KL divergence between the predicted and target pose distribution. The underlying principle unifies the existing approaches and resembles the attention mechanism. EPro-PnP significantly outperforms competitive baselines, closing the gap between PnP-based method and the taskspecific leaders on the LineMOD 6DoF pose estimation and nuScenes 3D object detection benchmarks. 3 * Part of work done during an internship at Alibaba Group.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: End-to-end Correspondence Learningmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

See 2 more Smart Citations

EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

Chen¹,

Wang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…M3DSSD [26] proposes a two-step feature alignment approach to overcome feature mismatching. MonoRUN [5] learns dense correspondences and geometry in a self-supervised manner with simple 3D bounding box annotations. Fig.…”

Section: Monocular 3d Object Detectionmentioning

confidence: 99%

Shape Prior Non-Uniform Sampling Guided Real-time Stereo 3D Object Detection

Gao,

Cao,

Pang

2021

Preprint

View full text Add to dashboard Cite

Pseudo-LiDAR based 3D object detectors have gained popularity due to their high accuracy. However, these methods need dense depth supervision and suffer from inferior speed. To solve these two issues, a recently introduced RTS3D builds an efficient 4D Feature-Consistency Embedding (FCE) space for the intermediate representation of object without depth supervision. FCE space splits the entire object region into 3D uniform grid latent space for feature sampling point generation, which ignores the importance of different object regions. However, we argue that, compared with the inner region, the outer region plays a more important role for accurate 3D detection. To encode more information from the outer region, we propose a shape prior non-uniform sampling strategy that performs dense sampling in outer region and sparse sampling in inner region. As a result, more points are sampled from the outer region and more useful features are extracted for 3D detection. Further, to enhance the feature discrimination of each sampling point, we propose a high-level semantic enhanced FCE module to exploit more contextual information and suppress noise better. Experiments on the KITTI dataset are performed to show the effectiveness of the proposed method. Compared with the baseline RTS3D, our proposed method has 2.57% improvement on AP 3d almost without extra network parameters. Moreover, our proposed method outperforms the state-of-the-art methods without extra supervision at a real-time speed.

show abstract

Multi-view LiDAR Guided Monocular 3D Object Detection

Liu

2022

Pattern Recognition and Computer Vision

View full text Add to dashboard Cite

MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation

Cited by 94 publications

References 29 publications

EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

Shape Prior Non-Uniform Sampling Guided Real-time Stereo 3D Object Detection

Multi-view LiDAR Guided Monocular 3D Object Detection

Contact Info

Product

Resources

About