Offboard 3D Object Detection from Point Cloud Sequences

Qi, Charles R.; Zhou, Yin; Najibi, Mahyar; Sun, Pei; Vo, Khoa; Deng, Boyang; Anguelov, Dragomir

doi:10.1109/cvpr46437.2021.00607

Cited by 129 publications

(71 citation statements)

References 91 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another motivation for using BEV features to perform perception tasks is that BEV is a desirable bridge to connect temporal and spatial space. For the human visual perception system, temporal information plays a crucial role in inferring the motion state of objects and identifying occluded objects, and many works in vision fields have demonstrated the effectiveness of using video data [2,27,26,33,19]. However, the existing state-of-the-art multi-camera 3D detection methods rarely exploit temporal information.…”

Section: Introductionmentioning

confidence: 99%

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

Li¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

3D visual perception tasks, including 3D detection and map segmentation based on multi-camera images, are essential for autonomous driving systems. In this work, we present a new framework termed BEVFormer, which learns unified BEV representations with spatiotemporal transformers to support multiple autonomous driving perception tasks. In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries. To aggregate spatial information, we design a spatial cross-attention that each BEV query extracts the spatial features from the regions of interest across camera views. For temporal information, we propose a temporal self-attention to recurrently fuse the history BEV information. Our approach achieves the new state-of-the-art 56.9% in terms of NDS metric on the nuScenes test set, which is 9.0 points higher than previous best arts and on par with the performance of LiDAR-based baselines. We further show that BEVFormer remarkably improves the accuracy of velocity estimation and recall of objects under low visibility conditions. The code will be released at https://github.com/zhiqi-li/BEVFormer.

show abstract

Section: Introductionmentioning

confidence: 99%

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

Li¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…An alternative approach by Deuge et al [31] introduced an unsupervised feature learning approach for outdoor object classification by projecting 3D LiDAR scans into 2D depth images. Recently, Qi et al [35] introduced an off-board (e.g. cloud) pipeline to get rid of the computational power limitation of on-board (i.e.…”

Section: A Object Detectionmentioning

confidence: 99%

3D ToF LiDAR in Mobile Robotics: A Review

Yang¹,

Li²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the past ten years, the use of 3D Time-of-Flight (ToF) LiDARs in mobile robotics has grown rapidly. Based on our accumulation of relevant research, this article systematically reviews and analyzes the use 3D ToF LiDARs in research and industrial applications. The former includes object detection, robot localization, long-term autonomy, LiDAR data processing under adverse weather conditions, and sensor fusion. The latter encompasses service robots, assisted and autonomous driving, and recent applications performed in response to public health crises. We hope that our efforts can effectively provide readers with relevant references and promote the deployment of existing mature technologies in real-world systems.

show abstract

“…[57] generates multi-frame object proposal features from the object feature vectors in the memory using a nonlocal attention block for aligning features. [58] processes LiDAR point cloud sequences for automatic labeling. A 3D multi-object tracker is used to match single-frame detections.…”

Section: Challenges and Open Fieldsmentioning

confidence: 99%

A Survey of Robust 3D Object Detection Methods in Point Clouds

Zimmer¹,

Erçelik²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

The purpose of this work is to review the stateof-the-art LiDAR-based 3D object detection methods, datasets, and challenges. We describe novel data augmentation methods, sampling strategies, activation functions, attention mechanisms, and regularization methods. Furthermore, we list recently introduced normalization methods, learning rate schedules and loss functions. Moreover, we also cover advantages and limitations of 10 novel autonomous driving datasets. We evaluate novel 3D object detectors on the KITTI, nuScenes, and Waymo dataset and show their accuracy, speed, and robustness. Finally, we mention the current challenges in 3D object detection in LiDAR point clouds and list some open issues. I. INTRODUCTIONAutonomous driving (AD) is increasingly gaining attention worldwide and can lead to many advantages to the population. The potential of this technology is clear, and it is predicted that it will dramatically change the transportation sector. The application of robust 3D Object Detection in autonomous driving is vital. In this context, robustness means to cope with detection errors (e.g., occlusions) and erroneous input (e.g., sensor noise). Sometimes, the environment is not optimal for detecting objects (e.g., in rain, snow, fog or bright sunlight). Robust detection stands for a good generalization to detect unseen objects in a different environment.Besides, infrastructure sensors (like the ones in the Providentia [1] system) support the perception of the environment, improve traffic safety, and offer higher robustness and performance through different mounting positions. Multiple view points help to better detect objects in 3D (position, orientation, and size), and to increase the robustness against sunlight, snow, dust, and fog.The aim of this survey paper is to provide an overview of novel 3D object detection methods and tricks. The best LiDAR-based 3D object detection algorithm on the KITTI dataset is 68.63% more accurate (on the 3D car class) than the best camera-only 3D object detection algorithm on that dataset. At night, the LiDAR performs better and is able to detect objects within a range of 50-60 m (Ouster OS1-64 gen. 2 LiDAR).Our contribution is partitioned into the following:• We provide state-of-the-art LiDAR-based 3D object detectors and show their accuracy, speed, and robustness.

show abstract

Offboard 3D Object Detection from Point Cloud Sequences

Cited by 129 publications

References 91 publications

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

3D ToF LiDAR in Mobile Robotics: A Review

A Survey of Robust 3D Object Detection Methods in Point Clouds

Contact Info

Product

Resources

About