A General Pipeline for 3D Detection of Vehicles

Du, Xinxin; Ang, Marcelo H.; Karaman, Sertaç; Rus, Daniela

doi:10.1109/icra.2018.8461232

Cited by 152 publications

(91 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LiDAR 3D Detection: The use of LiDAR data has proven to be essential input for SOTA frameworks [9,12,13,24,31,35,41] for 3D object detection applied to urban scenes.…”

Section: Related Workmentioning

confidence: 99%

M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

Brazil

Liu

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

491

349

View full text Add to dashboard Cite

Understanding the world in 3D is a critical component of urban autonomous driving. Generally, the combination of expensive LiDAR sensors and stereo RGB imaging has been paramount for successful 3D object detection algorithms, whereas monocular image-only methods experience drastically reduced performance. We propose to reduce the gap by reformulating the monocular 3D detection problem as a standalone 3D region proposal network. We leverage the geometric relationship of 2D and 3D perspectives, allowing 3D boxes to utilize well-known and powerful convolutional features generated in the image-space. To help address the strenuous 3D parameter estimations, we further design depth-aware convolutional layers which enable location specific feature development and in consequence improved 3D scene understanding. Compared to prior work in monocular 3D detection, our method consists of only the proposed 3D region proposal network rather than relying on external networks, data, or multiple stages. M3D-RPN is able to significantly improve the performance of both monocular 3D Object Detection and Bird's Eye View tasks within the KITTI urban autonomous driving dataset, while efficiently using a shared multi-class model.

show abstract

“…LiDAR 3D Detection: The use of LiDAR data has proven to be essential input for SOTA frameworks [9,12,13,24,31,35,41] for 3D object detection applied to urban scenes.…”

Section: Related Workmentioning

confidence: 99%

M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

Brazil

Liu

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

491

349

View full text Add to dashboard Cite

show abstract

“…The existing camera and LiDAR fusion methods can be divided into three different groups: 2D to 3D, proposal fusion, and dense fusion. In 2D to 3D approaches [5,19,21,30], 2D object detection is first performed on the RGB images using methods such as [17,23]. Afterwards, these 2D boxes are converted to 3D boxes using the LiDAR data.…”

Section: Object Detectionmentioning

confidence: 99%

Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

Meyer

Charland

Hegde

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

144

View full text Add to dashboard Cite

In this paper, we present an extension to LaserNet, an efficient and state-of-the-art LiDAR based 3D object detector. We propose a method for fusing image data with the LiDAR data and show that this sensor fusion method improves the detection performance of the model especially at long ranges. The addition of image data is straightforward and does not require image labels. Furthermore, we expand the capabilities of the model to perform 3D semantic segmentation in addition to 3D object detection. On a large benchmark dataset, we demonstrate our approach achieves state-of-the-art performance on both object detection and semantic segmentation while maintaining a low runtime.

show abstract

“…Several methods for object detection in autonomous driving scenes use lidar sensor data alongside images from an aligned camera [10], [11], [17], using the KITTI data set for training and evaluation [13]. While the KITTI data set is an excellent benchmark, its annotations are 3D bounding boxes, and object classes are limited to driving-relevant objects such as pedestrians, cars, and cyclists.…”

Section: A Deep Learning On Point Cloudsmentioning

confidence: 99%

“…The camera provides rich semantic information about object classes and facilitates the application of deep neural networks, and the lidar sensor provides precise 3D spatial measurements. Additionally, by identifying objects in 3D at the point level, rather than outputting rectangular 3D bounding boxes around detected objects [9], [16], [10], [11], [17], LDLS allows much more precise localization of object instances which do not neatly fit into rectangular boxes, such as people, animals, etc.…”

mentioning

confidence: 99%

LDLS: 3-D Object Segmentation Through Label Diffusion From 2-D Images

Wang

Chao

Wang

et al. 2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Object segmentation in 3D point clouds is a critical task for robots capable of 3D perception. Despite the impressive performance of deep learning-based approaches on object segmentation in 2D images, deep learning has not been applied nearly as successfully for 3D point cloud segmentation. Deep networks generally require large amounts of labeled training data, which are readily available for 2D images but are difficult to produce for 3D point clouds. In this paper, we present Label Diffusion Lidar Segmentation (LDLS), a novel approach for 3D point cloud segmentation which leverages 2D segmentation of an RGB image from an aligned camera to avoid the need for training on annotated 3D data. We obtain 2D segmentation predictions by applying Mask-RCNN to the RGB image, and then link this image to a 3D lidar point cloud by building a graph of connections among 3D points and 2D pixels. This graph then directs a semi-supervised label diffusion process, where the 2D pixels act as source nodes that diffuse object label information through the 3D point cloud, resulting in a complete 3D point cloud segmentation. We conduct empirical studies on the KITTI benchmark data set and on a mobile robot, demonstrating wide applicability and superior performance of LDLS compared to the previous state-of-the-art in 3D point cloud segmentation, without any need for either 3D training data or fine-tuning of the 2D image segmentation model.

show abstract

A General Pipeline for 3D Detection of Vehicles

Cited by 152 publications

References 38 publications

M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

LDLS: 3-D Object Segmentation Through Label Diffusion From 2-D Images

Contact Info

Product

Resources

About