nuScenes: A multimodal dataset for autonomous driving

Caesar, Holger; Bankiti, Varun; Lang, Alex H.; Vora, Sourabh; Liong, Venice Erin; Xu, Qian; Krishnan, Anush; Baldan, Giancarlo; Beijbom, Oscar

doi:10.48550/arxiv.1903.11027

Cited by 291 publications

(628 citation statements)

References 62 publications

(115 reference statements)

Supporting

Mentioning

626

Contrasting

Order By: Relevance

“…However, SC3D uses exhaustive search to generate numerous candidate proposals for template matching, so it performs well with few training samples. For the nuScenes [6] dataset, we directly apply the models, trained on the corresponding categories of the KITTI dataset, to evaluate performance on the nuScenes dataset. Specifically, the corresponding categories between KITTI and nuScenes datasets are Car→Car, Pedestrian→Pedestrian, Van→Truck, and Cyclist→Bicycle, respectively.…”

Section: Resultsmentioning

confidence: 99%

“…We develop a voxel-to-BEV target localization network, which can accurately detect the 3D target's center in the dense BEV space compared to sparse 3D space. Extensive results show that our method has achieved new state-of-the-art results on the KITTI dataset [20], and has a good generalization ability on the nuScenes [6] dataset.…”

Section: Introductionmentioning

confidence: 93%

“…Datasets. For 3D single object tracking, we use KITTI [20] and nuScenes [6] datasets for training and evaluation. Since the ground truth of the test set of KITTI dataset cannot be obtained, we follow [21,52] and use the training set to train and evaluate our method.…”

Section: Experimental Settingsmentioning

confidence: 99%

See 2 more Smart Citations

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Hui¹,

Wang²,

Cheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

3D object tracking in point clouds is still a challenging problem due to the sparsity of LiDAR points in dynamic environments. In this work, we propose a Siamese voxel-to-BEV tracker, which can significantly improve the tracking performance in sparse 3D point clouds. Specifically, it consists of a Siamese shape-aware feature learning network and a voxel-to-BEV target localization network. The Siamese shape-aware feature learning network can capture 3D shape information of the object to learn the discriminative features of the object so that the potential target from the background in sparse point clouds can be identified. To this end, we first perform template feature embedding to embed the template's feature into the potential target and then generate a dense 3D shape to characterize the shape information of the potential target. For localizing the tracked target, the voxel-to-BEV target localization network regresses the target's 2D center and the z-axis center from the dense bird's eye view (BEV) feature map in an anchor-free manner. Concretely, we compress the voxelized point cloud along z-axis through max pooling to obtain a dense BEV feature map, where the regression of the 2D center and the z-axis center can be performed more effectively. Extensive evaluation on the KITTI and nuScenes datasets shows that our method significantly outperforms the current state-of-the-art methods by a large margin. Code is available at https: //github.com/fpthink/V2B.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 93%

See 1 more Smart Citation

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Hui¹,

Wang²,

Cheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Those two setups were chosen to provide a diverse perception input to the ego vehicle, in terms of environment constellations and object types. In addition to that, we have tested position errors with the NuScenes dataset [21] as an example of non-simulated LiDAR information. Each scenario duration was sufficiently long for the environment to contain more than 1000 relevant object states in scope, in order to guarantee statistical significance.…”

Section: A Experimental Setupmentioning

confidence: 99%

“…Finally, we also explore the error detection capability of our monitor with real LiDAR data from the NuScenes dataset [21]. Since no ground truth velocity data is provided here, we restrict ourselves to the analysis of permanent position faults with sensor checks only.…”

Section: F Position Error Of a Real Sensormentioning

confidence: 99%

Fault-Tolerant Perception for Automated Driving A Lightweight Monitoring Approach

Buerkle¹,

Geißler²,

Paulitsch³

et al. 2021

Preprint

View full text Add to dashboard Cite

While the most visible part of the safety verification process of automated vehicles concerns the planning and control system, it is often overlooked that safety of the latter crucially depends on the fault-tolerance of the preceding environment perception. Modern perception systems feature complex and often machine-learning-based components with various failure modes that can jeopardize the overall safety. At the same time, a verification by for example redundant execution is not always feasible due to resource constraints. In this paper, we address the need for feasible and efficient perception monitors and propose a lightweight approach that helps to protect the integrity of the perception system while keeping the additional compute overhead minimal. In contrast to existing solutions, the monitor is realized by a well-balanced combination of sensor checks -here using LiDAR information -and plausibility checks on the object motion history. It is designed to detect relevant errors in the distance and velocity of objects in the environment of the automated vehicle. In conjunction with an appropriate planning system, such a monitor can help to make safe automated driving feasible.

show abstract

A survey of deep learning techniques for autonomous driving

Grigorescu

Trasnea

Cocias

et al. 2019

Journal of Field Robotics

1,214

495

View full text Add to dashboard Cite

The last decade witnessed increasingly rapid progress in self‐driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence (AI). The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving. We start by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration, and motion control algorithms. We investigate both the modular perception‐planning‐action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources, and computational hardware. The comparison presented in this survey helps gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices.

show abstract

nuScenes: A multimodal dataset for autonomous driving

Cited by 291 publications

References 62 publications

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Fault-Tolerant Perception for Automated Driving A Lightweight Monitoring Approach

A survey of deep learning techniques for autonomous driving

Contact Info

Product

Resources

About