Real-Time Foreground Segmentation for Surveillance Applications in NRCS Lidar Sequences

Abstract

Abstract. In this paper, we propose a point-level foreground-background separation technique for the segmentation of measurement sequences of a Non-repetitive Circular Scanning (NRCS) Lidar sensor, which is used as a 3D surveillance camera mounted in a fixed position. We show that by applying the NRCS Lidar technology, we can overcome various limitations of rotating multi-beam Lidar sensors, such as low vertical measurement resolution, which is disadvantageous in surveillance applications. As the main challenge, we need to efficiently balance between the spatial and the temporal resolution of the recorded range data. For this reason, we automatically generate and maintain a very high-resolution background model of the sensor’s Field of View, while for enabling real-time analysis of dynamic objects we use low integration time to extract the consecutive time frames. As a result, the laser reflections from foreground objects reflect sparse, but geometrically accurate samples of the silhouettes providing valuable input for higher-level shape description or event analysis steps. We demonstrate the efficiency of the new approach in different realistic NRCS Lidar measurements sequences, obtaining a 0.76 overall F1-score on the measured dataset.

LidPose: Real-Time 3D Human Pose Estimation in Sparse Lidar Point Clouds with Non-Repetitive Circular Scanning Pattern

“…The human pose estimation task can be applied in surveillance applications, which demand real-time solutions. To address this need, our approach involves transforming the representation of the NRCS lidar point cloud from 3D Cartesian coordinates to a spherical polar coordinate system, similar to our previous works in [ 39 , 40 ]. We generate a 2D pixel grid by discretizing the horizontal and vertical FoVs, where each 3D point’s distance from the sensor is mapped to a pixel determined by corresponding azimuth and elevation values.…”

“…More specifically, while the center of the field of view is scanned in every rotation of the pattern, outer regions are sampled less frequently, as demonstrated in Figure 2 . This particular, inhomogeneous point density distribution makes it difficult to apply existing lidar point cloud processing approaches on NRCS lidar measurement sequences [ 39 ]. Note that apart from depth data, the sensor also records the reflection intensity of the laser beams in the range 0–100% according to the Lambertian reflection model [ 33 ].…”

ST-DepthNet: A Spatio-Temporal Deep Network for Depth Completion Using a Single Non-Repetitive Circular Scanning Lidar

“…Range images are widely used, compact representations of Lidar-based depth measurements [1], [3], [16], which enable to adopt 2D convolution operations and effective image-based neural network architectures [4], [6] during processing.…”

“…In our experiments, we exploit the parameters of the Livox AVIA state-of-the-art NRCS Lidar sensor [3]. The sensor's FoV is mapped onto a 400×400 pixel lattice, which resolution (5.6 px/ • ) yields both high spatial accuracy and reasonable computational requirements.…”

“…1). The main challenge is here to efficiently balance between the spatial and the temporal resolution of the recorded range data using a suitable integration window [3].…”