Vote-Based 3D Object Detection with Context Modeling and SOB-3DNMS

Xie, Qian; Lai, Yuekun; Wu, Jing; Wang, Zhoutao; Zhang, Yiming; Xu, Kai; Wang, Jun

doi:10.1007/s11263-021-01456-w

Cited by 22 publications

(6 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the basis of VoteNet [109], Xie et al [49], [110], for the first time, introduce the self-attention mechanism of Transformers into the task of 3D object detection in indoor scenes. They propose the Multi-Level Context VoteNet (MLCVNet) to improve detection performance by encoding contextual information.…”

Section: D Tasksmentioning

confidence: 99%

Transformers in 3D Point Clouds: A Survey

Lu¹,

Quan²,

Wei³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, Transformer models have been proven to have the remarkable ability of long-range dependencies modeling. They have achieved satisfactory results both in Natural Language Processing (NLP) and image processing. This significant achievement sparks great interest among researchers in 3D point cloud processing to apply them to various 3D tasks. Due to the inherent permutation invariance and strong global feature learning ability, 3D Transformers are well suited for point cloud processing and analysis. They have achieved competitive or even better performance compared to the state-of-the-art non-Transformer algorithms. This survey aims to provide a comprehensive overview of 3D Transformers designed for various tasks (e.g. point cloud classification, segmentation, object detection, and so on). We start by introducing the fundamental components of the general Transformer and providing a brief description of its application in 2D and 3D fields. Then, we present three different taxonomies (i.e., Transformer implementation-based taxonomy, data representation-based taxonomy, and task-based taxonomy) for method classification, which allows us to analyze involved methods from multiple perspectives. Furthermore, we also conduct an investigation of 3D self-attention mechanism variants designed for performance improvement. To demonstrate the superiority of 3D Transformers, we compare the performance of Transformer-based algorithms in terms of point cloud classification, segmentation, and object detection. Finally, we point out three potential future research directions, expecting to provide some benefit references for the development of 3D Transformers.

show abstract

Section: D Tasksmentioning

confidence: 99%

Transformers in 3D Point Clouds: A Survey

Lu¹,

Quan²,

Wei³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The environment consists of multiple sensors for identifying moving objects, static objects, and obstacles. However, the RGB-D images and LiDAR images have many noises due to environmental factors which lead to image degradation hence it needs to be removed for obtaining high-quality images and cloud points for object detection [7], [8].…”

Section: Index Terms -3d Object Detectionnoise Removalsemantic Segmen...mentioning

confidence: 99%

YOLOv4Tiny: Bearing Angle Based Pose Estimation and Semantic Segmentation For 3D Object Detection From LiDAR Point Cloud & RGB-D Data

Rajendran

B.S.

2022

Preprint

View full text Add to dashboard Cite

3D object detection is a computer vision task that is used for various applications. However, it has several drawbacks due to occlusion, noise, inefficient field of view, etc. In addition, the recent LiDAR point clouds-based 3D object detection methods lack sparsity and homogeneity which affects the detection accuracy. In this paper, we detect 3D objects based on static and dynamic states by performing preprocessing, segmentation, extraction of features, classification, and regression considering RGB-D images, and LiDAR point clouds. Firstly, the quality of the image is enhanced by performing preprocessing which consists of two levels. In the first level, noise removal is performed for both RGB-D images and LiDAR cloud points by implementing Thresholding based Adaptive Median Filtering (TAMF) algorithm. In the second level, we perform point to voxel conversion for the LiDAR point clouds and fuse both RGB-D images and 3D LiDAR point clouds for better segmentation. Secondly, object segmentation is performed using Improved Mask R-CNN algorithm which creates masked instant bounding boxes by performing semantic segmentation considering both point and local-based information. Estimating pose for the segmented objects using bearing angle-based affine transformation method to extract features efficiently. Thirdly, Feature extraction is performed from the segmented 3D objects by considering high and low-level features using Attention-based YOLO V4 Tiny network and fusing both features for reducing the feature redundancy. Finally, based on the extracted feature the objects are classified as static and dynamic, and 3D bounding boxes are generated to improve the 3D object detection accuracy. The proposed work is evaluated by KITTI dataset. MATLAB R2020a simulation tool is used to perform simulation for the proposed work. The performance of the 3D object detection is evaluated in terms of several performance metrics such as accuracy, precision, recall, F-measure, computation time, and ROC-AUC curve compared to existing methods.

show abstract

“…MLCVNet [30] exploits multi-level contextual information and fuses them to vote for the object center. MLCVNet++ [31] further improves the network by proposing 3DNMS to remove redundant detections during post-processing. Yan et al [32] combine texture information from RGB data and geometric information from point cloud data, and use the deep Hough voting algorithm to suggest objects.…”

Section: Related Workmentioning

confidence: 99%

Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

Chu¹,

Deng²,

Yao³

et al. 2021

Preprint

View full text Add to dashboard Cite

As cameras are increasingly deployed in new application domains such as autonomous driving, performing 3D object detection on monocular images becomes an important task for visual scene understanding. Recent advances on monocular 3D object detection mainly rely on the "pseudo-LiDAR" generation, which performs monocular depth estimation and lifts the 2D pixels to pseudo 3D points. However, depth estimation from monocular images, due to its poor accuracy, leads to inevitable position shift of pseudo-LiDAR points within the object. Therefore, the predicted bounding boxes may suffer from inaccurate location and deformed shape. In this paper, we present a novel neighbor-voting method that incorporates neighbor predictions to ameliorate object detection from severely deformed pseudo-LiDAR point clouds. Specifically, each feature point around the object forms their own predictions, and then the "consensus" is achieved through voting. In this way, we can effectively combine the neighbors' predictions with local prediction and achieve more accurate 3D detection. To further enlarge the difference between the foreground region of interest (ROI) pseudo-LiDAR points and the background points, we also encode the ROI prediction scores of 2D foreground pixels into the corresponding pseudo-LiDAR points. We conduct extensive experiments on the KITTI benchmark to validate the merits of our proposed method. Our results on the bird's eye view detection outperform the state-of-the-art performance by a large margin, especially for the "hard" level detection.

show abstract

Vote-Based 3D Object Detection with Context Modeling and SOB-3DNMS

Cited by 22 publications

References 66 publications

Transformers in 3D Point Clouds: A Survey

Transformers in 3D Point Clouds: A Survey

YOLOv4Tiny: Bearing Angle Based Pose Estimation and Semantic Segmentation For 3D Object Detection From LiDAR Point Cloud & RGB-D Data

Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

Contact Info

Product

Resources

About

Vote-Based 3D Object Detection with Context Modeling and SOB-3DNMS

Cited by 22 publications

References 66 publications

Transformers in 3D Point Clouds: A Survey

Transformers in 3D Point Clouds: A Survey

YOLOv4Tiny: Bearing Angle Based Pose Estimation and Semantic Segmentation For 3D Object Detection From LiDAR Point Cloud &amp; RGB-D Data

Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

Contact Info

Product

Resources

About

YOLOv4Tiny: Bearing Angle Based Pose Estimation and Semantic Segmentation For 3D Object Detection From LiDAR Point Cloud & RGB-D Data