PillarGrid: Deep Learning-Based Cooperative Perception for 3D Object Detection from Onboard-Roadside LiDAR

Bai, Zhengwei; Wu, Guoyuan; Barth, Matthew; Liu, Yongkang; Sisbot, Emrah Akin; Oguchi, Kimio

doi:10.1109/itsc55140.2022.9921947

Cited by 35 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with the voxel-based approach, the pillar-based approach [3], [8], [9], [17], [19], [22] aims to reduce the time consumption during inference. These methods adjust the grid height to be equivalent to the height of the 3D space during the point cloud voxelization, thereby directly transforming the point cloud from the 3D shape to the 2D form in the BEV space.…”

Section: B Pillar-based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Grid-Based 3D Object Detection in Autonomous Driving With Improved Dimensionality Reduction

et al. 2023

View full text Add to dashboard Cite

Point cloud object detection is a pivotal technology in autonomous driving and robotics. Currently, the majority of cutting-edge point cloud detectors utilize Bird's Eye View (BEV) for detection, as it allows them to take advantage of well-explored 2D detection techniques. Nevertheless, dimensionality reduction of features from 3D space to BEV space unavoidably leads to information loss, and there is a lack of research on this issue. Existing methods typically obtain BEV features by collapsing voxel or point features along the height dimension via a pooling operation or convolution, resulting in a significant decrease in geometric information. To tackle this problem, we present a new point cloud backbone network for grid-based object detection, MDRNet, which is based on adaptive dimensionality reduction and multilevel spatial residual strategies. In MDRNet, the Spatial-aware Dimensionality Reduction (SDR) is designed to dynamically concentrate on the essential components of the object during 3D-to-BEV transformation. Moreover, the Multi-level Spatial Residuals (MSR) strategy is proposed to effectively fuse multi-level spatial information in BEV feature maps. Our MDRNet can be employed on any existing grid-based object detector, resulting in a remarkable improvement in performance. Numerous experiments conducted on nuScenes, KITTI and DAIR-V have shown that MDRNet surpasses existing SOTA approaches. In particular, on the nuScenes dataset, we attained an impressive 7.2% mAP and 5.0% NDS enhancement compared with CenterPoint.

show abstract

Section: B Pillar-based Methodsmentioning

confidence: 99%

“…Pillar-based PointPillar [3] Pooling Pillar-OD [8] Pooling MuRF-Net [9] Pooling InfoFocus [10] Pooling CenterPoint-pillar [13] Pooling WCNN3D [17] Pooling CVFNet [18] Pooling Pillargrid [19] Pooling PillarNet [22] Pooling…”

Section: Dimensionality Reductionmentioning

confidence: 99%

Enhancing Grid-Based 3D Object Detection in Autonomous Driving With Improved Dimensionality Reduction

et al. 2023

View full text Add to dashboard Cite

show abstract

“…However, most state-of-the-art object detection methods using point clouds are designed around a single onboard LiDAR, which is inherently limited by its range and susceptibility to occlusion, particularly in dense traffic scenarios. To address this, Bai et al [ 29 ] developed PillarGrid, a novel cooperative perception framework that combines information from multiple 3D LiDARs (both onboard and roadside) to enhance situational awareness for connected and automated vehicles (CAVs). PillarGrid comprises four key components: (i) cooperative preprocessing of point clouds, (ii) pillar-wise voxelization and feature extraction, (iii) grid-wise deep fusion of features from multiple sensors, and (iv) augmented 3D object detection using a convolutional neural network (CNN).…”

Section: Transition From Classical Perception To Collaborative Percep...mentioning

confidence: 99%

Collaborative Perception—The Missing Piece in Realizing Fully Autonomous Driving

Malik,

Khan,

Khan

et al. 2023

Sensors

View full text Add to dashboard Cite

Environment perception plays a crucial role in enabling collaborative driving automation, which is considered to be the ground-breaking solution to tackling the safety, mobility, and sustainability challenges of contemporary transportation systems. Despite the fact that computer vision for object perception is undergoing an extraordinary evolution, single-vehicle systems’ constrained receptive fields and inherent physical occlusion make it difficult for state-of-the-art perception techniques to cope with complex real-world traffic settings. Collaborative perception (CP) based on various geographically separated perception nodes was developed to break the perception bottleneck for driving automation. CP leverages vehicle-to-vehicle and vehicle-to-infrastructure communication to enable vehicles and infrastructure to combine and share information to comprehend the surrounding environment beyond the line of sight and field of view to enhance perception accuracy, lower latency, and remove perception blind spots. In this article, we highlight the need for an evolved version of the collaborative perception that should address the challenges hindering the realization of level 5 AD use cases by comprehensively studying the transition from classical perception to collaborative perception. In particular, we discuss and review perception creation at two different levels: vehicle and infrastructure. Furthermore, we also study the communication technologies and three different collaborative perception message-sharing models, their comparison analyzing the trade-off between the accuracy of the transmitted data and the communication bandwidth used for data transmission, and the challenges therein. Finally, we discuss a range of crucial challenges and future directions of collaborative perception that need to be addressed before a higher level of autonomy hits the roads.

show abstract

“…Predictive Radars take small data of the current scenes as input and perform general perception tasks such as PointPillars [70] for object detection and PointNet++ [71] for segmentation at first. The emerging cooperative perception [72][73][74][75] can greatly increase the perception range of each vehicle and deserves more attention in the future. Based on the results of general perception tasks, Predictive Radars will predict different future scenes which is the transition from small data to big data.…”

Section: Predictive Radarsmentioning

confidence: 99%

“…The high cost of collecting data and long-tail problem are serious issues. With the complete artificial systems in cyber space, Parallel Radars can generate sufficient virtual data to train new models for different downstream tasks such as object detection [26,27], segmentation [28][29][30][31], and cooperative perception [72][73][74][75], which can solve these problems effectively. Specific tasks such as the validation of new radars [94], super-resolution [95][96][97], and the analysis of radar placement [98,99], can also be settled.…”

Section: Autonomous Drivingmentioning

confidence: 99%

Parallel Radars: From Digital Twins to Digital Intelligence for Smart Radar Systems

Liu

Shen

Fan

et al. 2022

Sensors

View full text Add to dashboard Cite

Radar is widely employed in many applications, especially in autonomous driving. At present, radars are only designed as simple data collectors, and they are unable to meet new requirements for real-time and intelligent information processing as environmental complexity increases. It is inevitable that smart radar systems will need to be developed to deal with these challenges and digital twins in cyber-physical systems (CPS) have proven to be effective tools in many aspects. However, human involvement is closely related to radar technology and plays an important role in the operation and management of radars; thus, digital twins’ radars in CPS are insufficient to realize smart radar systems due to the inadequate consideration of human factors. ACP-based parallel intelligence in cyber-physical-social systems (CPSS) is used to construct a novel framework for smart radars, called Parallel Radars. A Parallel Radar consists of three main parts: a Descriptive Radar for constructing artificial radar systems in cyberspace, a Predictive Radar for conducting computational experiments with artificial systems, and a Prescriptive Radar for providing prescriptive control to both physical and artificial radars to complete parallel execution. To connect silos of data and protect data privacy, federated radars are proposed. Additionally, taking mines as an example, the application of Parallel Radars in autonomous driving is discussed in detail, and various experiments have been conducted to demonstrate the effectiveness of Parallel Radars.

show abstract

PillarGrid: Deep Learning-Based Cooperative Perception for 3D Object Detection from Onboard-Roadside LiDAR

Cited by 35 publications

References 21 publications

Enhancing Grid-Based 3D Object Detection in Autonomous Driving With Improved Dimensionality Reduction

Enhancing Grid-Based 3D Object Detection in Autonomous Driving With Improved Dimensionality Reduction

Collaborative Perception—The Missing Piece in Realizing Fully Autonomous Driving

Parallel Radars: From Digital Twins to Digital Intelligence for Smart Radar Systems

Contact Info

Product

Resources

About