V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

Li, Yiming; Ma, Dekun; An, Ziyan; Wang, Zixun; Zhong, Yiqi; Chen, Siheng; Chen, Feng

doi:10.48550/arxiv.2202.08449

Cited by 4 publications

(5 citation statements)

References 32 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…OPV2V [50] builds the first-of-a-kind 3D cooperative detection dataset using CARLA and OpenCDA co-simulation. V2XSet [49] and V2X-Sim [20] further explore the viability of vehicle-to-everything (V2X) perception using synthesized data generated from CARLA simulator [11]. Unlike the above-simulated datasets, DAIR-V2X is the first realworld dataset for cooperative detection.…”

Section: Autonomous Driving Datasetsmentioning

confidence: 99%

“…1, for example, the ego vehicle (red liDAR) struggles to perceive the upcoming objects located across the way due to occlusions. Incorporating the LiDAR features from the nearby CAV (green scans) can largely broaden the sensing range of the vehicle and make it even see across the [50] 2022 Sim V2V -44k 11k Yes 230k 1 CARLA V2X-Sim [20] 2022 Sim V2V&I -60K 10k Yes 26.6k 1 CARLA V2XSet [49] 2022 Sim V2V&I -44K 11k Yes 230k 1 CARLA DAIR-V2X [54] 2022 occluded corner.…”

Section: Introductionmentioning

confidence: 99%

“…Despite the great promise, however, it remains challenging to validate V2V perception in real-world scenarios due to the lack of public benchmarks. Most of the existing V2V datasets, including OPV2V [50], V2X-Sim [20], and V2XSet [49], rely on open-source simulators like CARLA [11] to generate synthetic road scenes and traffic dynamics with simulated connected vehicles. However, it is well known that there exists a clear domain gap between synthetic data and real-world data, as the traffic behavior and sensor rendering in simulators are often not realistic enough [27,37].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

Xu¹,

Xia²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

Modern perception systems of autonomous vehicles are known to be sensitive to occlusions and lack the capability of long perceiving range. It has been one of the key bottlenecks that prevents Level 5 autonomy. Recent research has demonstrated that the Vehicle-to-Vehicle (V2V) cooperative perception system has great potential to revolutionize the autonomous driving industry. However, the lack of a real-world dataset hinders the progress of this field. To facilitate the development of cooperative perception, we present V2V4Real, the first large-scale real-world multimodal dataset for V2V perception. The data is collected by two vehicles equipped with multi-modal sensors driving together through diverse scenarios. Our V2V4Real dataset covers a driving area of 410 km, comprising 20K LiDAR frames, 40K RGB frames, 240K annotated 3D bounding boxes for 5 classes, and HDMaps that cover all the driving routes. V2V4Real introduces three perception tasks, including cooperative 3D object detection, cooperative 3D object tracking, and Sim2Real domain adaptation for cooperative perception. We provide comprehensive benchmarks of recent cooperative perception algorithms on three tasks. The V2V4Real dataset and codebase can be found at research.seas.ucla.edu/mobility-lab/v2v4real.

show abstract

Section: Autonomous Driving Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

Xu¹,

Xia²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent works are also studying synthetic data generation for CP [14], [15]. For example, the open-source CARLA simulator [16] is used to simulate the traffic environment and generate synthetic data.…”

Section: Related Workmentioning

confidence: 99%

CoPEM: Cooperative Perception Error Models for Autonomous Driving

Piazzoni

Cherian

Vijay

et al. 2022

2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

In this paper, we introduce the notion of Cooperative Perception Error Models (coPEMs) towards achieving an effective and efficient integration of V2X solutions within a virtual test environment. We focus our analysis on the occlusion problem in the (onboard) perception of Autonomous Vehicles (AV), which can manifest as misdetection errors on the occluded objects. Cooperative perception (CP) solutions based on Vehicle-to-Everything (V2X) communications aim to avoid such issues by cooperatively leveraging additional points of view for the world around the AV. This approach usually requires many sensors, mainly cameras and LiDARs, to be deployed simultaneously in the environment either as part of the road infrastructure or on other traffic vehicles. However, implementing a large number of sensor models in a virtual simulation pipeline is often prohibitively computationally expensive. Therefore, in this paper, we rely on extending Perception Error Models (PEMs) to efficiently implement such cooperative perception solutions along with the errors and uncertainties associated with them. We demonstrate the approach by comparing the safety achievable by an AV challenged with a traffic scenario where occlusion is the primary cause of a potential collision.

show abstract

“…However, the lack of publicly available datasets with infrastructure integration slows down the progression of this field significantly. The first dataset to address this issue by combining multiple viewing angles was V2X-Sim in 2021 [15]. With the usage of CARLA-SUMO co-simulations, they created a synthetic dataset consisting of six RGB-, depth-and semantic cameras, one 360°LiDAR and GPS/IMU data on the vehicle side, and four of the same cameras and the same LiDAR attached to roadside units (RSU).…”

Section: B Vehicle-infrastructure Cooperation Datasetsmentioning

confidence: 99%

Reliability of Conventional Infrastructure in Context of Automated Driving illustrated by the SLAM Problem

et al. 2021

View full text Add to dashboard Cite

This paper we present our vision and ongoing work for a novel dataset designed to advance research into the interoperability of intelligent vehicles and infrastructure, specifically aimed at enhancing cooperative perception and interaction in the realm of public transportation. Unlike conventional datasets centered on ego-vehicle data, this approach encompasses both a stationary sensor tower and a moving vehicle, each equipped with cameras, LiDARs, and GNSS, while the vehicle additionally includes an inertial navigation system. Our setup features comprehensive calibration and time synchronization, ensuring seamless and accurate sensor data fusion crucial for studying complex, dynamic scenes. Emphasizing public transportation, the dataset targets to include scenes like bus station maneuvers and driving on dedicated bus lanes, reflecting the specifics of small public buses. We introduce the open-source .4mse file format for the new dataset, accompanied by a research kit. This kit provides tools such as ego-motion compensation or LiDAR-to-camera projection enabling advanced research on intelligent vehicle-infrastructure integration. Our approach does not include annotations; however, we plan to implement automatically generated labels sourced from state-of-the-art public repositories.Several aspects are still up for discussion, and timely feedback from the community would be greatly appreciated. A sneak preview on one data frame will be available at a Google Colab Notebook. Moreover, we will use the related GitHub Repository to collect remarks and suggestions.

show abstract

V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

Cited by 4 publications

References 32 publications

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

CoPEM: Cooperative Perception Error Models for Autonomous Driving

Reliability of Conventional Infrastructure in Context of Automated Driving illustrated by the SLAM Problem

Contact Info

Product

Resources

About