Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation

Ku, Jason S.; Pon, Alex D.; Walsh, Seán; Waslander, Steven L.

doi:10.1109/iros40897.2019.8968242

Cited by 27 publications

(3 citation statements)

References 43 publications

(82 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sparse Dd point cloud data is encoded with compact multi-view. Ku used a virtual multi-view synthesis method to generate a set of virtual views for each detected pedestrian [ 19 ]. These views are used to create accurate position estimates during training and reasoning.…”

Section: Realted Workmentioning

confidence: 99%

Pedestrian Detection with Multi-View Convolution Fusion Algorithm

Liu

Han

Zhang

et al. 2022

Entropy

View full text Add to dashboard Cite

In recent years, the pedestrian detection technology of a single 2D image has been dramatically improved. When the scene becomes very crowded, the detection performance will deteriorate seriously and cannot meet the requirements of autonomous driving perception. With the introduction of the multi-view method, the task of pedestrian detection in crowded or fuzzy scenes has been significantly improved and has become a widely used method in autonomous driving. In this paper, we construct a double-branch feature fusion structure, the first branch adopts a lightweight structure, the second branch further extracts features and gets the feature map obtained from each layer. At the same time, the receptive field is enlarged by expanding convolution. To improve the speed of the model, the keypoint is used instead of the entire object for regression without an NMS post-processing operation. Meanwhile, the whole model can be learned from end to end. Even in the presence of many people, the method can still perform better on accuracy and speed. In the standard of Wildtrack and MultiviewX dataset, the accuracy and running speed both perform better than the state-of-the-art model, which has great practical significance in the autonomous driving field.

show abstract

Section: Realted Workmentioning

confidence: 99%

Pedestrian Detection with Multi-View Convolution Fusion Algorithm

Liu

Han

Zhang

et al. 2022

Entropy

View full text Add to dashboard Cite

show abstract

“…MMF [20] benefits from multi-task learning and multi-sensor fusion. VMVS [16] generates a set of virtual views for each detected pedestrian in pseudo point clouds. Then the different views are used to produce an accurate orientation estimation.…”

Section: Related Workmentioning

confidence: 99%

Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion

Wu¹,

Peng²,

Xie³

et al. 2022

Preprint

View full text Add to dashboard Cite

Current LiDAR-only 3D detection methods inevitably suffer from the sparsity of point clouds. Many multi-modal methods are proposed to alleviate this issue, while different representations of images and point clouds make it difficult to fuse them, resulting in suboptimal performance. In this paper, we present a novel multi-modal framework SFD (Sparse Fuse Dense), which utilizes pseudo point clouds generated from depth completion to tackle the issues mentioned above. Different from prior works, we propose a new RoI fusion strategy 3D-GAF (3D Grid-wise Attentive Fusion) to make fuller use of information from different types of point clouds. Specifically, 3D-GAF fuses 3D RoI features from the pair of point clouds in a grid-wise attentive way, which is more fine-grained and more precise. In addition, we propose a SynAugment (Synchronized Augmentation) to enable our multi-modal framework to utilize all data augmentation approaches tailored to LiDAR-only methods. Lastly, we customize an effective and efficient feature extractor CPConv (Color Point Convolution) for pseudo point clouds. It can explore 2D image features and 3D geometric features of pseudo point clouds simultaneously. Our method holds the highest entry on the KITTI car 3D object detection leaderboard † , demonstrating the effectiveness of our SFD. Code will be made publicly available.

show abstract

“…As can be seen in Figure 2, the performance of approaches which have a focus on LiDAR for the final predictions (F-PointNet and PointPillars) have a significant drop in performance for long range when compared to approaches like AVOD which use both sensors. VMVS [13], STD [14], IPOD [12], F-ConvNet [15] and TANet [16] were not evaluated in detail, due to the lack of source code available or similarity to other approaches. VMVS, F-ConvNet and TANet are conceptually similar to AVOD, F-PointNet or PointPillars.…”

Section: Introductionmentioning

confidence: 99%

LRPD: Long Range 3D Pedestrian Detection Leveraging Specific Strengths of LiDAR and RGB

Fürst

Wasenmüller

Stricker

2020

2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

While short range 3D pedestrian detection is sufficient for emergency breaking, long range detections are required for smooth breaking and gaining trust in autonomous vehicles. The current state-of-the-art on the KITTI benchmark performs suboptimal in detecting the position of pedestrians at long range. Thus, we propose an approach specifically targeting long range 3D pedestrian detection (LRPD), leveraging the density of RGB and the precision of LiDAR. Therefore, for proposals, RGB instance segmentation and LiDAR point based proposal generation are combined, followed by a second stage using both sensor modalities symmetrically. This leads to a significant improvement in mAP on long range compared to the current state-of-the art. The evaluation of our LRPD approach was done on the pedestrians from the KITTI benchmark.

show abstract

Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation

Cited by 27 publications

References 43 publications

Pedestrian Detection with Multi-View Convolution Fusion Algorithm

Pedestrian Detection with Multi-View Convolution Fusion Algorithm

Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion

LRPD: Long Range 3D Pedestrian Detection Leveraging Specific Strengths of LiDAR and RGB

Contact Info

Product

Resources

About