Self-Supervised Learning for Domain Adaptation on Point Clouds

Achituve, Idan; Maron, Haggai; Chechik, Gal

doi:10.1109/wacv48630.2021.00017

Cited by 132 publications

(89 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most older works on input representation, which would make a good pre-training tasks, were focused on part-based and shape classification [30] [31] [32] [6] [33] or focused on registration [34], and are now a bit too unrelated to current autonomous driving scene understanding to make for a decent baseline.…”

Section: B 3d Transfer Learningmentioning

confidence: 99%

COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets

Sanchez¹,

Deschaud²,

Goulette³

2022

Preprint

View full text Add to dashboard Cite

Transfer learning is a proven technique in 2D computer vision to leverage the large amount of data available and achieve high performance with datasets limited in size due to the cost of acquisition or annotation. In 3D, annotation is known to be a costly task; nevertheless, transfer learning methods have only recently been investigated. Unsupervised pre-training has been heavily favored as no very large annotated dataset are available.In this work, we tackle the case of real-time 3D semantic segmentation of sparse outdoor LiDAR scans. Such datasets have been on the rise, but with different label sets even for the same task. In this work, we propose here an intermediate-level label set called the coarse labels, which allows all the data available to be leveraged without any manual labelization. This way, we have access to a larger dataset, alongside a simpler task of semantic segmentation. With it, we introduce a new pre-training task: the coarse label pre-training, also called COLA.We thoroughly analyze the impact of COLA on various datasets and architectures and show that it yields a noticeable performance improvement, especially when the finetuning task has access only to a small dataset.

show abstract

Section: B 3d Transfer Learningmentioning

confidence: 99%

COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets

Sanchez¹,

Deschaud²,

Goulette³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…More recently, several studies suggested using SSL techniques for point cloud understanding [10,14,23,36,45,47,48,51,58,62,65]. Example 3D pretext tasks includes orientation estimation [38], deformation reconstruction [1], geometric structural cues [52] and spatial cues [34,49]. Inspired by the jigsaw puzzles in images [35], [48] proposes to reconstruct point clouds from the randomly rearranged parts.…”

Section: Related Workmentioning

confidence: 99%

“…In this step, the sub-clouds {p i } g i=1 can be tokenized into point tokens {z i } g i=1 , relating to effective local geometric patterns. In our experiments, 1 Point tokens have two forms, discrete integer number and corresponding word embedding in V, which are equivalent.…”

Section: Point Embeddings a Naive Approach Treats Per Point Asmentioning

confidence: 99%

Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling

Nian¹,

Tang²,

Rao³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present Point-BERT, a new paradigm for learning Transformers to generalize the concept of BERT [8] to 3D point cloud. Inspired by BERT, we devise a Masked Point Modeling (MPM) task to pre-train point cloud Transformers. Specifically, we first divide a point cloud into several local point patches, and a point cloud Tokenizer with a discrete Variational AutoEncoder (dVAE) is designed to generate discrete point tokens containing meaningful local information. Then, we randomly mask out some patches of input point clouds and feed them into the backbone Transformers. The pre-training objective is to recover the original point tokens at the masked locations under the supervision of point tokens obtained by the Tokenizer. Extensive experiments demonstrate that the proposed BERT-style pretraining strategy significantly improves the performance of standard point cloud Transformers. Equipped with our pretraining strategy, we show that a pure Transformer architecture attains 93.8% accuracy on ModelNet40 and 83.1% accuracy on the hardest setting of ScanObjectNN, surpassing carefully designed point cloud models with much fewer hand-made designs. We also demonstrate that the representations learned by Point-BERT transfer well to new tasks and domains, where our models largely advance the state-of-the-art of few-shot point cloud classification task. The code and pre-trained models are available at https: //github.com/lulutang0608/Point-BERT.

show abstract

“…al. [1] introduces an additional selfsupervised reconstruction task to improve the classification performance on the target domain. [36] designs a sparse voxel completion network to perform point cloud comple-tion for domain adaptive semantic segmentation.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency

Luo

Cai²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Figure 1: Visualization of detection results for domain adaptation from KITTI to Waymo dataset. Left: Predictions of baseline model trained on KITTI dataset and directly tested on Waymo dataset. The model can classify and localize the objects, but produces inaccurate box scale due to geometric mismatch. The predicted boxes are therefore noticeably smaller than the ground truth. Right: Predictions of our domain-adaptive MLC-Net, which demonstrates accurate bounding box scale even though MLC-Net is trained without access to any target domain annotation or statistical information. Best viewed in color.

show abstract

Self-Supervised Learning for Domain Adaptation on Point Clouds

Cited by 132 publications

References 48 publications

COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets

COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets

Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling

Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency

Contact Info

Product

Resources

About