Learning 2D to 3D Lifting for Object Detection in 3D for Autonomous Vehicles

Srivastava, Siddharth; Jurie, Frédéric; Sharma, Gaurav

doi:10.1109/iros40897.2019.8967624

Cited by 43 publications

(14 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we want to work towards a more complete system that allows SLAM to succeed in videos with more challenges, e.g. motion blurs and rolling shutter distortion, as well as explore SfM [45], 3D object localization [46] and top-view mapping [13] in unconstrained scenarios.…”

Section: Discussionmentioning

confidence: 99%

Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM

Zhuang

Tran²,

Lee

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Self-calibration of camera intrinsics and radial distortion has a long history of research in the computer vision community. However, it remains rare to see real applications of such techniques to modern Simultaneous Localization And Mapping (SLAM) systems, especially in driving scenarios. In this paper, we revisit the geometric approach to this problem, and provide a theoretical proof that explicitly shows the ambiguity between radial distortion and scene depth when twoview geometry is used to self-calibrate the radial distortion.In view of such geometric degeneracy, we propose a learning approach that trains a convolutional neural network (CNN) on a large amount of synthetic data. We demonstrate the utility of our proposed method by applying it as a checkerboard-free calibration tool for SLAM, achieving comparable or superior performance to previous learning and hand-crafted methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM

Zhuang

Tran²,

Lee

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…A representative model is Mono3D (Chen et al, 2016), which uses additional instance and semantic segmentation along with further features to reason about the pose and location of objects in 3D space. Srivastava et al (2019) modify the generative adversarial network of BirdNet (Beltrán et al, 2018) to create a BEV projection from the monocular representation. All following operations such as feature extractions and predictions are then performed on this new representation.…”

Section: Informed Monocular Approachesmentioning

confidence: 99%

Survey and Systematization of 3D Object Detection Models and Methods

Moritz¹,

Friederich²,

Egger³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper offers a comprehensive survey of recent developments in 3D object detection covering the full pipeline from input data, over data representation and feature extraction to the actual detection modules. We include basic concepts, focus our survey on a broad spectrum of different approaches arising in the last ten years and propose a systematization which offers a practical framework to compare those approaches on the methods level.

show abstract

“…Robust models like VoxelNet, PointNet, and RoarNet were able to process 3D sensory data, combined video, and LiDAR information [30][31][32]. e popular methods of 3D detection can be roughly classified as Monocular Image-Based Methods, Monocular Image-Based Methods, and Fusion-Based Methods, which work by extrapolating 2D bounding boxes, generating the 3D representation of the point cloud fusing front view images and point clouds, respectively [29,33,34]. However, these methods are computationally expensive and require more time to execute as compared to 2D detectors.…”

Section: Object Detectionmentioning

confidence: 99%

Lightweight Object Detection Ensemble Framework for Autonomous Vehicles in Challenging Weather Conditions

Walambe

Marathe

Kotecha

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The computer vision systems driving autonomous vehicles are judged by their ability to detect objects and obstacles in the vicinity of the vehicle in diverse environments. Enhancing this ability of a self-driving car to distinguish between the elements of its environment under adverse conditions is an important challenge in computer vision. For example, poor weather conditions like fog and rain lead to image corruption which can cause a drastic drop in object detection (OD) performance. The primary navigation of autonomous vehicles depends on the effectiveness of the image processing techniques applied to the data collected from various visual sensors. Therefore, it is essential to develop the capability to detect objects like vehicles and pedestrians under challenging conditions such as like unpleasant weather. Ensembling multiple baseline deep learning models under different voting strategies for object detection and utilizing data augmentation to boost the models’ performance is proposed to solve this problem. The data augmentation technique is particularly useful and works with limited training data for OD applications. Furthermore, using the baseline models significantly speeds up the OD process as compared to the custom models due to transfer learning. Therefore, the ensembling approach can be highly effective in resource-constrained devices deployed for autonomous vehicles in uncertain weather conditions. The applied techniques demonstrated an increase in accuracy over the baseline models and were able to identify objects from the images captured in the adverse foggy and rainy weather conditions. The applied techniques demonstrated an increase in accuracy over the baseline models and reached 32.75% mean average precision (mAP) and 52.56% average precision (AP) in detecting cars in the adverse fog and rain weather conditions present in the dataset. The effectiveness of multiple voting strategies for bounding box predictions on the dataset is also demonstrated. These strategies help increase the explainability of object detection in autonomous systems and improve the performance of the ensemble techniques over the baseline models.

show abstract

Learning 2D to 3D Lifting for Object Detection in 3D for Autonomous Vehicles

Cited by 43 publications

References 31 publications

Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM

Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM

Survey and Systematization of 3D Object Detection Models and Methods

Lightweight Object Detection Ensemble Framework for Autonomous Vehicles in Challenging Weather Conditions

Contact Info

Product

Resources

About