Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection

Abstract

Self-driving cars must detect other vehicles and pedestrians in 3D to plan safe routes and avoid collisions. State-of-the-art 3D object detectors, based on deep learning, have shown promising accuracy but are prone to over-fit to domain idiosyncrasies, making them fail in new environmentsa serious problem if autonomous vehicles are meant to operate freely. In this paper, we propose a novel learning approach that drastically reduces this gap by fine-tuning the detector on pseudo-labels in the target domain, which our method generates while the vehicle is parked, based on replays of previously recorded driving sequences. In these replays, objects are tracked over time, and detections are interpolated and extrapolated-crucially, leveraging future information to catch hard cases. We show, on five autonomous driving datasets, that fine-tuning the object detector on these pseudo-labels substantially reduces the domain gap to new driving environments, yielding drastic improvements in accuracy and detection reliability.

ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection

“…The performance gaps between source only results and fully supervised oracle results are closed by a large percentage. Besides, we outperform existing approaches [11], [23], [24] by a notable margin (around 13% ∼ 17%) based on the same setup. It's also noteworthy that our approach even outperforms the oracle results for all categories on the Waymo → KITTI setting when further combined with target statistics [11] as shown in Fig.…”

“…As a result, when the detector is adapted to a sparse domain, it can not generate enough highquality pseudo labels to provide sufficient knowledge in the selftraining stage. TABLE 4 Unsupervised adaptation results of SF-UDA 3D [23], Dreaming [24], MLC-Net [26] and our ST3D++. We report AP 3D of car at IoU 0.7 and 0.5 on nuScenes → KITTI.…”

“…(vi) We conduct extensive experiments on four datasets for three categories. The proposed ST3D++ outperforms ST3D [25] over all categories on all adaptation settings as well as other existing UDA methods, SF-UDA 3D [23], Dreaming [24] and MLC-Net [26] by a large margin. (vii) We explore to leverage temporal information to further improve ST3D++ through fused sequential point frames, which also mitigates the point density gaps across domains.…”

“…Though the performance has been improved, the method needs the target statistics information, and its effectiveness depends on the source and target data distributions. SF-UDA 3D [23] and Dreaming [24] leveraged the extra tracker and time consistency regularization along target point cloud sequence to generate better pseudo labels for self-training. SRDAN [64] employed scale-aware and rangeaware feature adversarial alignment manners to match the distribution between two domains, which might suffer from stability and convergence issues.…”

See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation

“…Wang et al [56] proposed a semi-supervised approach using object-size statistics of the target domain to resize training samples in the labelled source domain. A popular approach is the use of self-training [43,63,64,67] with a focus on generating quality pseudo-labels using temporal information [43,67] or an IoU scoring criterion for historical pseudo-labels [63,64]. In particular, while Yang et al [63,64] has drastically improved the performance over previous works, it is not practical for a lidar that can adjust its scan pattern in real-time.…”