V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

Li, Yiming; Ma, Dekun; An, Ziyan; Wang, Zixun; Zhong, Yiqi; Chen, Siheng; Feng, Chen

doi:10.1109/lra.2022.3192802

Cited by 128 publications

(70 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…1, we can see that with our uncertainty quantification method, detected objects with low accuracy tend to have large uncertainties, and the constructed uncertainty set covers the ground-truth bounding box in most cases. Compared with the state-of-thearts [18], [19], our Double-M Quantification method achieves up to 4× improvement on uncertainty score and up to 3.13% accuracy improvement on the comprehensive collaborative perception dataset, V2X-SIM [1].…”

Section: Introductionmentioning

confidence: 95%

“…Multi-agent collaborative object detection has been proposed to leverage the viewpoints of other agents to improve the detection accuracy compared with the individual viewpoint [1]. Recent research has shown the effectiveness of early, late, and intermediate fusion of collaborative detection, which respectively transmits raw data, output bounding boxes, and intermediate features [2], [3], [4], and the improved collaborative object detection results will benefit the self-driving decisions of connected and autonomous vehicles (CAVs) [5].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Uncertainty Quantification of Collaborative Detection for Self-Driving

Su¹,

Li²,

He³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Sharing information between connected and autonomous vehicles (CAVs) fundamentally improves the performance of collaborative object detection for self-driving. However, CAVs still have uncertainties on object detection due to practical challenges, which will affect the later modules in self-driving such as planning and control. Hence, uncertainty quantification is crucial for safety-critical systems such as CAVs. Our work is the first to estimate the uncertainty of collaborative object detection. We propose a novel uncertainty quantification method, called Double-M Quantification, which tailors a moving block bootstrap (MBB) algorithm with direct modeling of the multivariant Gaussian distribution of each corner of the bounding box. Our method captures both the epistemic uncertainty and aleatoric uncertainty with one inference pass based on the offline Double-M training process. And it can be used with different collaborative object detectors. Through experiments on the comprehensive collaborative perception dataset, we show that our Double-M method achieves more than 4× improvement on uncertainty score and more than 3% accuracy improvement, compared with the state-of-the-art uncertainty quantification methods. Our code is public on https://coperception. github.io/double-m-quantification/.

show abstract

Section: Introductionmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

Uncertainty Quantification of Collaborative Detection for Self-Driving

Su¹,

Li²,

He³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We validate our SyncNet on LIDAR-based 3D object detection task [3,9] with a multi-agent dataset, V2X-Sim [14]. V2X-Sim is built with the co-simulation of SUMO [12] and CARLA [7].…”

Section: Multi-agent 3d Object Detection Datasetmentioning

confidence: 99%

“…We extensively evaluate the novel latency-aware collaborative perception system with SyncNet on V2X-Sim dataset [14] on collaborative 3D object detection for autonomous driving. The results verify the robustness of our system and show substantial improvements over state-of-the-art approaches.…”

Section: Introductionmentioning

confidence: 99%

Latency-Aware Collaborative Perception

Lei¹,

Ren²,

Hu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Collaborative perception has recently shown great potential to improve perception capabilities over single-agent perception. Existing collaborative perception methods usually consider an ideal communication environment. However, in practice, the communication system inevitably suffers from latency issues, causing potential performance degradation and high risks in safety-critical applications, such as autonomous driving. To mitigate the effect caused by the inevitable latency, from a machine learning perspective, we present the first latency-aware collaborative perception system, which actively adapts asynchronous perceptual features from multiple agents to the same time stamp, promoting the robustness and effectiveness of collaboration. To achieve such a featurelevel synchronization, we propose a novel latency compensation module, called SyncNet, which leverages feature-attention symbiotic estimation and time modulation techniques. Experiments results show that the proposed latency aware collaborative perception system with SyncNet can outperforms the state-of-the-art collaborative perception method by 15.6% in the communication latency scenario and keep collaborative perception being superior to single agent perception under severe latency.

show abstract

“…Sooner or later, these autonomous systems will be deployed on roads at scale, opening up opportunities for cooperation between them. Previous works in [4], [5], [6], [7], [8], [9], [10], [11], [12] have demonstrated that by leveraging the Vehicle-to-Everything (V2X) communication technology, AVs and infrastructure can perform cooperative perception by using the shared sensing information and thus significantly enhance the perception performance. Despite the remarkable improvement, these works evaluate the proposed systems on the dataset with natural scenarios that do not contain sufficient safety-critical scenes.…”

Section: Introductionmentioning

confidence: 99%

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

Xiang¹,

Xu²,

Xia³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advancements in Vehicle-to-Everything communication technology have enabled autonomous vehicles to share sensory information to obtain better perception performance. With the rapid growth of autonomous vehicles and intelligent infrastructure, the V2X perception systems will soon be deployed at scale, which raises a safety-critical question: how can we evaluate and improve its performance under challenging traffic scenarios before the real-world deployment? Collecting diverse large-scale real-world test scenes seems to be the most straightforward solution, but it is expensive and timeconsuming, and the collections can only cover limited scenarios. To this end, we propose the first open adversarial scene generator V2XP-ASG that can produce realistic, challenging scenes for modern LiDAR-based multi-agent perception system. V2XP-ASG learns to construct an adversarial collaboration graph and simultaneously perturb multiple agents' poses in an adversarial and plausible manner. The experiments demonstrate that V2XP-ASG can effectively identify challenging scenes for a large range of V2X perception systems. Meanwhile, by training on the limited number of generated challenging scenes, the accuracy of V2X perception systems can be further improved by 12.3% on challenging and 4% on normal scenes.

show abstract

V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

Cited by 128 publications

References 48 publications

Uncertainty Quantification of Collaborative Detection for Self-Driving

Uncertainty Quantification of Collaborative Detection for Self-Driving

Latency-Aware Collaborative Perception

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

Contact Info

Product

Resources

About