Geometry-based Distance Decomposition for Monocular 3D Object Detection

Shi, Xuepeng; Ye, Qi; Chen, Xiaozhi; Chen, Chuangrong; Chen, Zhixiang; Kim, Taekyun

doi:10.48550/arxiv.2104.03775

Cited by 6 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The learning procedure is thus called auxiliary learning, in contrast to multitask learning, for which all tasks in training will be of interest in inference too. Auxiliary tasks and auxiliary learning have shown many successful applications in computer vision (Zhang et al 2014;Mordan et al 2018;Liu, Davison, and Johns 2019;Ye et al 2021;Valada, Radwan, and Burgard 2018). Although simple, exploiting comprehensive 2D auxiliary tasks has not been studied well in monocular 3D object detection.…”

Section: Related Work and Our Contributionsmentioning

confidence: 99%

“…3D object detection is a critical component in many computer vision applications in practice, such as autonomous driving and robot navigation. High performing methods often require more costly system setups such as Lidar sensors (Yan, Mao, and Li 2018;Lang et al 2019;Qi et al 2019; for precise depth measurements or stereo cameras (Li, Chen, and Shen 2019;Qin, Wang, and Lu 2019;Xu 30 35 3D Detection Performance (APR40|IoU≥0.7) MonoRCNN (Shi et al 2021) DDMP3D (Wang et al 2021a) CaDDN (Reading et al 2021) MonoEF (Zhou et al 2021) MonoFlex (Zhang, Lu, and Zhou 2021) GUPNet (Lu et al 2021) MonoCon (Ours) Sun et al 2020) for stereo depth estimation, and are often more computationally expensive. To alleviate those "burden" and due to the potential prospects of reduced cost and increased modular redundancy, monocular 3D object detection that aims to localize 3D object bounding boxes from an input 2D image has emerged as a promising alternative approach with much attention received in the computer vision and AI community (Chen et al 2016;Manhardt, Kehl, and Gaidon 2019;Simonelli et al 2019;Brazil and Liu 2019;Wang et al 2020;Ye et al 2020;Shi, Chen, and Kim 2020;Luo et al 2021;Kumar, Brazil, and Liu 2021;Wang et al 2021c,d).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

Liu

Xue

2022

AAAI

View full text Add to dashboard Cite

Monocular 3D object detection aims to localize 3D bounding boxes in an input single 2D image. It is a highly challenging problem and remains open, especially when no extra information (e.g., depth, lidar and/or multi-frames) can be leveraged in training and/or inference. This paper proposes a simple yet effective formulation for monocular 3D object detection without exploiting any extra information. It presents the MonoCon method which learns Monocular Contexts, as auxiliary tasks in training, to help monocular 3D object detection. The key idea is that with the annotated 3D bounding boxes of objects in an image, there is a rich set of well-posed projected 2D supervision signals available in training, such as the projected corner keypoints and their associated offset vectors with respect to the center of 2D bounding box, which should be exploited as auxiliary tasks in training. The proposed MonoCon is motivated by the Cramer–Wold theorem in measure theory at a high level. In implementation, it utilizes a very simple end-to-end design to justify the effectiveness of learning auxiliary monocular contexts, which consists of three components: a Deep Neural Network (DNN) based feature backbone, a number of regression head branches for learning the essential parameters used in the 3D bounding box prediction, and a number of regression head branches for learning auxiliary contexts. After training, the auxiliary context regression branches are discarded for better inference efficiency. In experiments, the proposed MonoCon is tested in the KITTI benchmark (car, pedestrian and cyclist). It outperforms all prior arts in the leaderboard on the car category and obtains comparable performance on pedestrian and cyclist in terms of accuracy. Thanks to the simple design, the proposed MonoCon method obtains the fastest inference speed with 38.7 fps in comparisons. Our code is released at https://git.io/MonoCon.

show abstract

Section: Related Work and Our Contributionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

Liu

Xue

2022

AAAI

View full text Add to dashboard Cite

show abstract

“…For 2D representation-based 3D detector, an intuitive solution is to leverage a 2D object detector [3], [4]. Similar to 2D object detectors, prior work [8], [9], [11], [14], [15], [16], [56], [57], [58], [59] directly estimates 3D bounding boxes from camera images and relies on perspective modeling of the 2D projected object and its 3D objects. On the other hand, depth supervision via point clouds or depth maps is accessible during training.…”

Section: Lidar-based 3d Detectionmentioning

confidence: 99%

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

Chen¹,

Huang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Camera-based 3D object detectors are welcome due to their wider deployment and lower price than LiDAR sensors. We revisit the prior stereo modeling DSGN about the stereo volume constructions for representing both 3D geometry and semantics. We polish the stereo modeling and propose our approach, DSGN++, aiming for improving information flow throughout the 2D-to-3D pipeline in the following three main aspects. First, to effectively lift the 2D information to stereo volume, we propose depth-wise plane sweeping (DPS) that allows denser connections and extracts depth-guided features. Second, for better grasping differently spaced features, we present a novel stereo volume -Dual-view Stereo Volume (DSV) that integrates front-view and top-view features and reconstructs sub-voxel depth in the camera frustum. Third, as the foreground region becomes less dominant in 3D space, we firstly propose a multi-modal data editing strategy -Stereo-LiDAR Copy-Paste, which ensures cross-modal alignment and improves data efficiency. Without bells and whistles, extensive experiments in various modality setups on the popular KITTI benchmark show that our method consistently outperforms other camera-based 3D detectors for all categories. Code will be released at https://github.com/chenyilun95/DSGN2.

show abstract

“…cos(δ), are zero, which could stop backpropagation learning at the extreme opposite output. To resolve this problem, researchers [50,51,52,53,54,55,56,57,58,59,60,61] performed regression on both the cosine and sine of the angle (cos(θ), sin(θ)). When converting any given 1D Scalar-Based Orientation θ into 2-Dimensional values, they can be mapped to 2-D Cartesian system with angle's cosine value projected to x-axis and sine value projected to y-axis.…”

Section: Alpha (Local/allocentric Rotation)mentioning

confidence: 99%

SoK: Vehicle Orientation Representations for Deep Rotation Estimation

Tu¹,

Peng²,

Leung³

et al. 2021

Preprint

View full text Add to dashboard Cite

In recent years, there is an influx of deep learning models for 3D vehicle object detection. However, little attention was paid to orientation prediction. Existing research work proposed various vehicle orientation representation methods for deep learning, however a holistic, systematic review has not been conducted. Through our experiments, we categorize and compare the accuracy performance of various existing orientation representations using the KITTI 3D object detection dataset [1], and propose a new form of orientation representation: Tricosine. Among these, the 2D Cartesian-based representation [cos(θ), sin(θ)], or Single Bin, achieves the highest accuracy, with additional channeled inputs (positional encoding and depth map) not boosting prediction performance. Our code is published on GitHub: https://github.com/umd-fire-coml/KITTI-orientationlearning

show abstract

Geometry-based Distance Decomposition for Monocular 3D Object Detection

Cited by 6 publications

References 31 publications

Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

SoK: Vehicle Orientation Representations for Deep Rotation Estimation

Contact Info

Product

Resources

About