Behind the Curtain: Learning Occluded Shapes for 3D Object Detection

Abstract

Advances in LiDAR sensors provide rich 3D data that supports 3D scene understanding. However, due to occlusion and signal miss, LiDAR point clouds are in practice 2.5D as they cover only partial underlying shapes, which poses a fundamental challenge to 3D perception. To tackle the challenge, we present a novel LiDAR-based 3D object detection model, dubbed Behind the Curtain Detector (BtcDet), which learns the object shape priors and estimates the complete object shapes that are partially occluded (curtained) in point clouds. BtcDet first identifies the regions that are affected by occlusion and signal miss. In these regions, our model predicts the probability of occupancy that indicates if a region contains object shapes and integrates this probability map with detection features and generates high-quality 3D proposals. Finally, the occupancy estimation is integrated into the proposal refinement module to generate accurate bounding boxes. Extensive experiments on the KITTI Dataset and the Waymo Open Dataset demonstrate the effectiveness of BtcDet. Particularly for the 3D detection of both cars and cyclists on the KITTI benchmark, BtcDet surpasses all of the published state-of-the-art methods by remarkable margins. Code is released.

DNN Filter for Bias Reduction in Distribution-to-Distribution Scan Matching

“…If the geometic approach was prone to uniformly distributed errors for any scene, there would be no reason to employ an alternate registration technique (and thus lose useful info). As is discussed in [15,27], however, certain geometric properties in a scene violate the assumption of a static scene significantly enough to introduce systemic bias. This highlights the inherent tradeoff between solution accuracy and solution integrity.…”

“…Analytical techniques permit the direct calculation of error bounds for solution covariance by considering the cost function at convergence [4,22] or directly via a least-squares analysis of the PDF occupying each voxel [14]. While these registration techniques boast impressive accuracy prediction on static scenes, motion of the ego-vehicle can introduce systemic bias through rangeshadowing [15], self-occlusion [27], and "rolling shutter" motion distortion [9]. For scan registration algorithms to be useful in real-world applications, they must maintain robustness in the presence of these forms of bias.…”

“…Partial overlap between point clouds, which occurs in regions subject to range-shadowing and self-occlusion, presents a problem for registration. Range-shadowing tends to inject systemic bias across a large swath of voxels in a constant direction normal to the edge of each shadow [15], while perspective shift bias is highly localized on the surfaces of objects that appear different from various angles [27]. Geometric strategies exist to reduce the bias of rangeshadowing in D2D algorithms, either through ground plane removal [19,7] or through a spherical coordinate representation [9,15,5], however, error introduced by self-occlusion is significantly more complex.…”

“…Self-supervised methods overcome partial overlap by jointly learning geometric representations, keypoint detection, and correspondence [25,30]. BTCdet augments preliminary estimates from a backbone network by filling in points on occluded surfaces using a network trained a priori on commonly encountered object shapes [27].…”

Improving 3D Vulnerable Road User Detection With Point Augmentation

“…Another approach to improve detection on occluded, distant or small instances is to reconstruct the points of missing shapes. With the aid of point completion algorithms, a generative module is trained to predict the full shape of objects from incomplete point sets in a self-contained manner or through external datasets [13]- [16]. The instances with incomplete geometry are replaced or augmented with the generated shapes to increase the confidence of predictions for small and occluded objects.…”