Combining LiDAR space clustering and convolutional neural networks for pedestrian detection

Abstract

Pedestrian detection is an important component for safety of autonomous vehicles, as well as for traffic and street surveillance. There are extensive benchmarks on this topic and it has been shown to be a challenging problem when applied on real use-case scenarios. In purely image-based pedestrian detection approaches, the state-of-the-art results have been achieved with convolutional neural networks (CNN) and surprisingly few detection frameworks have been built upon multi-cue approaches. In this work, we develop a new pedestrian detector for autonomous vehicles that exploits LiDAR data, in addition to visual information. In the proposed approach, LiDAR data is utilized to generate region proposals by processing the three dimensional point cloud that it provides. These candidate regions are then further processed by a state-of-the-art CNN classifier that we have fine-tuned for pedestrian detection. We have extensively evaluated the proposed detection process on the KITTI dataset. The experimental results show that the proposed LiDAR space clustering approach provides a very efficient way of generating region proposals leading to higher recall rates and fewer misses for pedestrian detection. This indicates that LiDAR data can provide auxiliary information for CNN-based approaches.

The Use of Three-Dimensional Convolutional Neural Networks to Interpret LiDAR for Forest Inventory

“…One study implemented a 3D CNN for use in airborne LiDAR to identify helicopter landing zones in real time [36]. Others have used them in conjunction with terrestrial LiDAR to map obstacles for autonomous cars [37,38]. One common application has been to identify malignancies using 3D medical scans [39][40][41].…”

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

“…Many works [61], [91], [99]- [101], [106], [108], [109], [111], [117]- [120], [123] deal with the 2D object detection problem on the front-view 2D image plane. Compared to 2D detection, 3D detection is more challenging since the object's distance to the ego-vehicle needs to be estimated.…”

“…Processing the sparse data via clustering (e.g. [100], [106]- [108]) or 3D CNN (e.g. [29], [136]) is usually very timeconsuming and infeasible for online autonomous driving.…”

“…Several works on multi-modal object detection cluster and segment 3D LiDAR points to generate 3D region proposals (e.g. [100], [106], [108]). Still, they use a LiDAR 2D representation to extract features for fusion; (3).…”

Person-in-WiFi: Fine-Grained Person Perception Using WiFi

“…Moreover, LiDARs have sampling rate in the range of 5-20 Hz, which is much lower than other sensors such as cameras or WiFi adapters (100 Hz). To increase robustness, many researchers combine LiDAR with RGB cameras [36,30,17] or with motion sensors [11] for pedestrian detection.…”