PillarFlowNet: A Real-time Deep Multitask Network for LiDAR-based 3D Object Detection and Scene Flow Estimation

Duffhauss, Fabian; Baur, Stefan A.

doi:10.1109/iros45743.2020.9341002

Cited by 6 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motion detection methods based on point cloud registration, combining geometric and structural features with neural network-based interests, have enhanced the detection and tracking of moving objects [24]. Novel multi-task models have been introduced for simultaneous scene flow estimation and object detection, achieving significant improvements in performance and latency [45]. These models represent a crucial advancement in the field, combining two complex tasks into a single efficient process.…”

Section: D Object Detectionmentioning

confidence: 99%

Kalman-Based Scene Flow Estimation for Point Cloud Densification and 3D Object Detection in Dynamic Scenes

Ding,

Zhang,

et al. 2024

Sensors

View full text Add to dashboard Cite

Point cloud densification is essential for understanding the 3D environment. It provides crucial structural and semantic information for downstream tasks such as 3D object detection and tracking. However, existing registration-based methods struggle with dynamic targets due to the incompleteness and deformation of point clouds. To address this challenge, we propose a Kalman-based scene flow estimation method for point cloud densification and 3D object detection in dynamic scenes. Our method effectively tackles the issue of localization errors in scene flow estimation and enhances the accuracy and precision of shape completion. Specifically, we introduce a Kalman filter to correct the dynamic target’s position while estimating long sequence scene flow. This approach helps eliminate the cumulative localization error during the scene flow estimation process. Extended experiments on the KITTI 3D tracking dataset demonstrate that our method significantly improves the performance of LiDAR-only detectors, achieving superior results compared to the baselines.

show abstract

Section: D Object Detectionmentioning

confidence: 99%

Kalman-Based Scene Flow Estimation for Point Cloud Densification and 3D Object Detection in Dynamic Scenes

Ding,

Zhang,

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Any Motion Detector (AMD-Net), a class agnostic scene dynamics estimation architecture proposed in [32], outputs a static and dynamic segmentation of each cell and its velocity. Instead of handling object detection and scene flow estimation separately, the PillaFlowNet network [33] takes both simultaneously, providing fast and accurate results. A Fast Hierarchical Network (FH-Net) method is proposed in [34], minimizing the cost and latency issues of typical scene flow methods and directly computing the key points flow from the source point to its target points.…”

Section: Related Workmentioning

confidence: 99%

AI-Enabled Condition Monitoring Framework for Outdoor Mobile Robots Using 3D LiDAR Sensor

Pookkuttath,

Palanisamy,

Elara

2023

Mathematics

View full text Add to dashboard Cite

An automated condition monitoring (CM) framework is essential for outdoor mobile robots to trigger prompt maintenance and corrective actions based on the level of system deterioration and outdoor uneven terrain feature states. Vibration indicates system failures and terrain abnormalities in mobile robots; hence, five vibration threshold classes for CM in outdoor mobile robots were identified, considering both vibration source system deterioration and uneven terrain. This study proposes a novel CM approach for outdoor mobile robots using a 3D LiDAR, employed here instead of its usual use as a navigation sensor, by developing an algorithm to extract the vibration-indicated data based on the point cloud, assuring low computational costs without losing vibration characteristics. The algorithm computes cuboids for two prominent clusters in every point cloud frame and sets motion points at the corners and centroid of the cuboid. The three-dimensional vector displacement of these points over consecutive point cloud frames, which corresponds to the vibration-affected clusters, are compiled as vibration indication data for each threshold class. A simply structured 1D Convolutional Neural Network (1D CNN)-based vibration threshold prediction model is proposed for fast, accurate, and real-time application. Finally, a threshold class mapping framework is developed which fuses the predicted threshold classes on the 3D occupancy map of the workspace, generating a 3D CbM map in real time, fostering a Condition-based Maintenance (CbM) strategy. The offline evaluation test results show an average accuracy of vibration threshold classes of 89.6% and consistent accuracy during real-time field case studies of 89%. The test outcomes validate that the proposed 3D-LiDAR-based CM framework is suitable for outdoor mobile robots, assuring the robot’s health and operational safety.

show abstract

“…The PillarFlowNet method has been developed using a single network to perform multiple tasks in real time [175]. The computational load caused by the use of separate deep neural networks in normal methods is reduced in PillarFlowNet thanks to the single network.…”

Section: Pillarflownetmentioning

confidence: 99%

Deep Learning Based Point Cloud Processing Techniques

Hazer

Yıldırım

2022

IEEE Access

View full text Add to dashboard Cite

In this study, deep learning techniques and algorithms used in point cloud processing have been analysed. Methods, technical properties and algorithms developed for 3D Object Classification and Segmentation, 3D object detection and tracking and 3D scene flow of point cloud data have been also analysed. 3D point cloud sensing techniques have been grouped as Multi-view, Volumetric approach and raw point cloud processing and mathematical models of them have been analysed. In 3D Object Classification and Segmentation, algorithms are given by analysing it in different categories as Convolutional Neural Network (CNN) based, Graph based, Hierarchical Data Structure-Based Methods and Others. 3D object detection and tracking, Segmentation-based, Frustum-based, Discretization-based analysed as point based and other methods. In each section, deep learning algorithms are compared with respect to applicability to real-time processing, amount of points, and relevance to large-scale or small-scale areas. In the last section, comparisons of point cloud processing methods are made and their advantages and disadvantages are given in the form of a table.

show abstract

PillarFlowNet: A Real-time Deep Multitask Network for LiDAR-based 3D Object Detection and Scene Flow Estimation

Cited by 6 publications

References 24 publications

Kalman-Based Scene Flow Estimation for Point Cloud Densification and 3D Object Detection in Dynamic Scenes

Kalman-Based Scene Flow Estimation for Point Cloud Densification and 3D Object Detection in Dynamic Scenes

AI-Enabled Condition Monitoring Framework for Outdoor Mobile Robots Using 3D LiDAR Sensor

Deep Learning Based Point Cloud Processing Techniques

Contact Info

Product

Resources

About