MonoEF: Extrinsic Parameter Free Monocular 3D Object Detection

Zhou, Yunsong; He, Yuan; Zhu, Hongzi; Wang, Cheng; Li, Hongyang; Jiang, Qinhong

doi:10.1109/tpami.2021.3136899

Cited by 14 publications

(5 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…standard monocular detectors [16,18,22,41,62] take as input only a single image and mostly adopt center-guided pipelines following conventional 2D detectors [38,44,60]. M3D-RPN [2] designs a depth-aware convolution along with 3D anchors to generate better 3D region proposals.…”

Section: Related Workmentioning

confidence: 99%

MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformer

Zhou¹,

Zhu²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Mobile monocular 3D object detection (Mono3D) (e.g., on a vehicle, a drone, or a robot) is an important yet challenging task. Existing transformer-based offline Mono3D models adopt grid-based vision tokens, which is suboptimal when using coarse tokens due to the limited available computational power. In this paper, we propose an online Mono3D framework, called MonoATT, which leverages a novel vision transformer with heterogeneous tokens of varying shapes and sizes to facilitate mobile Mono3D. The core idea of MonoATT is to adaptively assign finer tokens to areas of more significance before utilizing a transformer to enhance Mono3D. To this end, we first use prior knowledge to design a scoring network for selecting the most important areas of the image, and then propose a token clustering and merging network with an attention mechanism to gradually merge tokens around the selected areas in multiple stages. Finally, a pixel-level feature map is reconstructed from heterogeneous tokens before employing a SOTA Mono3D detector as the underlying detection core. Experiment results on the real-world KITTI dataset demonstrate that MonoATT can effectively improve the Mono3D accuracy for both near and far objects and guarantee low latency. MonoATT yields the best performance compared with the state-of-the-art methods by a large margin and is ranked number one on the KITTI 3D benchmark.

show abstract

Section: Related Workmentioning

confidence: 99%

MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformer

Zhou¹,

Zhu²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Researchers have been improving detection performance via upgrading detection frameworks (Wang et al 2021d;Liu et al 2019), losses (Simonelli et al 2020) and non-maximum suppression (Kumar, Brazil, and Liu 2021) as well as performing joint detection and 3D reconstruction . To reduce 2D to 3D ambiguities, various strategies have been developed, e.g., Pseudo-LiDAR (Wang et al 2019;Ma et al 2019Ma et al , 2020Simonelli et al 2021;Park et al 2021), novel convolutions (Ding et al 2020) and backbones (Kumar et al 2022), considering camera geometry (Zhou et al 2021), using shape models Chabot et al 2017) and leveraging videos (Brazil et al 2020).…”

Section: Related Workmentioning

confidence: 99%

RADIANT: Radar-Image Association Network for 3D Object Detection

Long

Kumar

Morris

et al. 2023

AAAI

View full text Add to dashboard Cite

As a direct depth sensor, radar holds promise as a tool to improve monocular 3D object detection, which suffers from depth errors, due in part to the depth-scale ambiguity. On the other hand, leveraging radar depths is hampered by difficulties in precisely associating radar returns with 3D estimates from monocular methods, effectively erasing its benefits. This paper proposes a fusion network that addresses this radar-camera association challenge. We train our network to predict the 3D offsets between radar returns and object centers, enabling radar depths to enhance the accuracy of 3D monocular detection. By using parallel radar and camera backbones, our network fuses information at both the feature level and detection level, while at the same time leveraging a state-of-the-art monocular detection technique without retraining it. Experimental results show significant improvement in mean average precision and translation error on the nuScenes dataset over monocular counterparts. Our source code is available at https://github.com/longyunf/radiant.

show abstract

“…MonoDETR [54] constructed a foreground depth map to help the model understand the scene object depth spatial distributions with the aid of the DETR-liked model structure [55]. MonoEF [56], [57] and MoGDE [58] proposed to model the camera extrinsic changes and provided a grounded depth carrying the camera information, making the model achieve extremely accurate performance. For perspective projection: Ivan et al [11] combined the keypoint method and the projection to do geometry reasoning.…”

Section: B Monocular 3d Object Detectionmentioning

confidence: 99%

Geometry Uncertainty Projection Network for Monocular 3D Object Detection

Yang

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

161

View full text Add to dashboard Cite

Geometry plays a significant role in monocular 3D object detection. It can be used to estimate object depth by using the perspective projection between object's physical size and 2D projection in the image plane, which can introduce mathematical priors into deep models. However, this projection process also introduces error amplification, where the error of the estimated height is amplified and reflected into the projected depth. It leads to unreliable depth inferences and also impairs training stability.To tackle this problem, we propose a novel Geometry Uncertainty Propagation Network (GUPNet++) by modeling geometry projection in a probabilistic manner. This ensures depth predictions are well-bounded and associated with a reasonable uncertainty. The significance of introducing such geometric uncertainty is twofold: (1). It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning. (2). It can be derived to a highly reliable confidence to indicate the quality of the 3D detection result, enabling more reliable detection inference. Experiments show that the proposed approach not only obtains (state-of-the-art) SOTA performance in image-based monocular 3D detection but also demonstrates superiority in efficacy with a simplified framework. The code and model will be released at https://github.com/SuperMHP/GUPNet Plus.

show abstract

MonoEF: Extrinsic Parameter Free Monocular 3D Object Detection

Cited by 14 publications

References 56 publications

MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformer

MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformer

RADIANT: Radar-Image Association Network for 3D Object Detection

Geometry Uncertainty Projection Network for Monocular 3D Object Detection

Contact Info

Product

Resources

About