EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection

Huang, Tong; Liu, Zhe; Chen, Xiwu; Bai, Xiang

doi:10.1007/978-3-030-58555-6_3

Cited by 327 publications

(208 citation statements)

References 40 publications

(71 reference statements)

Supporting

Mentioning

206

Contrasting

Order By: Relevance

“…To tackle the irregular data format of point clouds, most existing works project the point clouds to regular grids to be processed by 2D or 3D CNN. The pioneer work MV3D [9] projects the point clouds to 2D bird view grids and places lots of predefined 3D anchors for generating 3D bounding boxes, and the following works [11], [13], [17], [56], [57], [58] develop better strategies for multi-sensor fusion. [12], [14], [15] introduce more efficient frameworks with bird-eye view representation while [59] proposes to fuse grid features of multiple scales.…”

Section: D Object Detection With Point Cloudsmentioning

confidence: 99%

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Shi

Guo²,

Jiang³

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,580

1,045

View full text Add to dashboard Cite

3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields. In this paper, we propose the Point-Voxel Region based Convolution Neural Networks (PV-RCNNs) for accurate 3D detection from point clouds. First, we propose a novel 3D object detector, PV-RCNN-v1, which employs the voxel-to-keypoint scene encoding and keypoint-to-grid RoI feature abstraction two novel steps. These two steps deeply incorporate both 3D voxel CNN and PointNet-based set abstraction for learning discriminative point-cloud features. Second, we propose a more advanced framework, PV-RCNN-v2, for more efficient and accurate 3D detection. It consists of two major improvements, where the first one is the sectorized proposal-centric strategy for efficiently producing more representative and uniformly distributed keypoints, and the second one is the VectorPool aggregation to replace set abstraction for better aggregating local point-cloud features with much less resource consumption. With these two major modifications, our PV-RCNN-v2 runs more than twice as fast as the v1 version while still achieving better performance on the large-scale Waymo Open Dataset with 150m × 150m detection range. Extensive experiments demonstrate that our proposed PV-RCNNs significantly outperform previous state-of-the-art 3D detection methods on both the Waymo Open Dataset and the highly-competitive KITTI benchmark.

show abstract

Section: D Object Detection With Point Cloudsmentioning

confidence: 99%

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Shi

Guo²,

Jiang³

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

1,580

1,045

View full text Add to dashboard Cite

show abstract

“…In the process of 3D object detection, inconsistency between the localization and classification confidence is a critical issue [75]. To solve the problem, a consistency enforcing loss is utilized to increase the consistency of both the localization and classification in EPNet [76]. Moreover, the point features is enhanced with semantic image features in a point-wise manner without image annotations.…”

Section: Multi-sensor Fusion-based 3d Object Detection Methodsmentioning

confidence: 99%

A Review of 3D Object Detection for Autonomous Driving of Electric Vehicles

Dai

Chen

Bao

et al. 2021

WEVJ

View full text Add to dashboard Cite

In recent years, electric vehicles have achieved rapid development. Intelligence is one of the important trends to promote the development of electric vehicles. As a result, autonomous driving system is becoming one of the core systems of electric vehicles. Considering that environmental perception is the basis of intelligent planning and safe decision-making for intelligent vehicles, this paper presents a survey of the existing perceptual methods in vehicles, especially 3D object detection, which guarantees the reliability and safety of vehicles. In this review, we first introduce the role of perceptual module in autonomous driving system and a relationship with other modules. Then, we classify and analyze the corresponding perception methods based on the different sensors. Finally, we compare the performance of the surveyed works on public datasets and discuss the possible future research interests.

show abstract

“…So, it needs the point-wise feature extraction and the cross-level fusion strategy. EPNet [199] (shown in Figure 9) introduces a novel LI-Fusion module which is a point-wise fusion manner enhancing the point clouds feature map with the image feature. The LI-Fusion includes a point-wise correspondence generation part and a LiDAR-guided fusion part.…”

Section: Cross-level Fusion Strategymentioning

confidence: 99%

The Fusion Strategy of 2D and 3D Information Based on Deep Learning: A Review

et al. 2021

View full text Add to dashboard Cite

Recently, researchers have realized a number of achievements involving deep-learning-based neural networks for the tasks of segmentation and detection based on 2D images, 3D point clouds, etc. Using 2D and 3D information fusion for the advantages of compensation and accuracy improvement has become a hot research topic. However, there are no critical reviews focusing on the fusion strategies of 2D and 3D information integration based on various data for segmentation and detection, which are the basic tasks of computer vision. To boost the development of this research domain, the existing representative fusion strategies are collected, introduced, categorized, and summarized in this paper. In addition, the general structures of different kinds of fusion strategies were firstly abstracted and categorized, which may inspire researchers. Moreover, according to the methods included in this paper, the 2D information and 3D information of different methods come from various kinds of data. Furthermore, suitable datasets are introduced and comparatively summarized to support the relative research. Last but not least, we put forward some open challenges and promising directions for future research.

show abstract

EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection

Cited by 327 publications

References 40 publications

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

A Review of 3D Object Detection for Autonomous Driving of Electric Vehicles

The Fusion Strategy of 2D and 3D Information Based on Deep Learning: A Review

Contact Info

Product

Resources

About