Self-Supervised Point Cloud Representation Learning with Occlusion Auto-Encoder

Zhou, Junsheng; Xu, Wen; Baorui, Ma,; Liu, Yu‐Shen; Gao, Yue; Fang, Y.; Han, Zhitao

doi:10.48550/arxiv.2203.14084

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As LiDAR sensors become more affordable and common, there has been an increasing amount of research interest in self-supervised learning on 3D point clouds. Previous works proposed to learn representations of object or scene level point clouds through contrastive learning [8,25,30] or reconstruction [23,26,28,33], which is useful in downstream classification or segmentation tasks. In contrast, our supervision signals come from the unique structure of the human body and our learned backbone is particularly useful in downstream human keypoint estimation tasks.…”

Section: Self-supervised Learning For Point Cloudsmentioning

confidence: 99%

RSF: Optimizing Rigid Scene Flow From 3D Point Clouds Without Labels

Deng

Zakhor

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Training a 3D human keypoint detector from point clouds in a supervised manner requires large volumes of high quality labels. While it is relatively easy to capture large amounts of human point clouds, annotating 3D keypoints is expensive, subjective, error prone and especially difficult for long-tail cases (pedestrians with rare poses, scooterists, etc.). In this work, we propose GC-KPL -Geometry Consistency inspired Key Point Leaning, an approach for learning 3D human joint locations from point clouds without human labels. We achieve this by our novel unsupervised loss formulations that account for the structure and movement of the human body. We show that by training on a large training set from Waymo Open Dataset [21] without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach. Further, the backbone benefits from the unsupervised training and is useful in downstream fewshot learning of keypoints, where fine-tuning on only 10 percent of the labeled training data gives comparable performance to fine-tuning on the entire set. We demonstrated that GC-KPL outperforms by a large margin over SoTA when trained on entire dataset and efficiently leverages large volumes of unlabeled data.* Work done as an intern at Waymo.

show abstract

Section: Self-supervised Learning For Point Cloudsmentioning

confidence: 99%

RSF: Optimizing Rigid Scene Flow From 3D Point Clouds Without Labels

Deng

Zakhor

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…It is particularly important for 3D point cloud analysis, since the collection and annotation of point cloud data are much more expensive than 2D images. Popular SSL methods for point cloud include reconstruction [68,16,75,60,8,25,76,70,34,40,73,65,15], instance contrastive feature learning [49,51], consistency feature learning against augmentations [31], and other pretext tasks [52,43,1]. Among these methods, the masked auto-encoding [60,76,70,40] has been receiving more and more attention recently.…”

Section: Related Workmentioning

confidence: 99%

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

Zhang¹,

Shi²,

Wu³

2022

SSRN Journal

View full text Add to dashboard Cite

“…Classic optimizationbased methods [15,3,28,29] tried to resolve this problem by inferring continuous surfaces from the geometry of point clouds. With the rapid development of deep learning [63,33,56,61,27,24,54,55,58,53], the neural networks have shown great potential in reconstructing 3D surfaces [30,9,8,22,13,17,51,43,31,52]. In the following, we will briefly review the studies of deep learning based methods.…”

Section: Related Workmentioning

confidence: 99%

Learning Consistency-Aware Unsigned Distance Functions Progressively from Raw Point Clouds

Zhou¹,

Baorui²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Surface reconstruction for point clouds is an important task in 3D computer vision. Most of the latest methods resolve this problem by learning signed distance functions (SDF) from point clouds, which are limited to reconstructing shapes or scenes with closed surfaces. Some other methods tried to represent shapes or scenes with open surfaces using unsigned distance functions (UDF) which are learned from large scale ground truth unsigned distances. However, the learned UDF is hard to provide smooth distance fields near the surface due to the noncontinuous character of point clouds. In this paper, we propose a novel method to learn consistency-aware unsigned distance functions directly from raw point clouds. We achieve this by learning to move 3D queries to reach the surface with a field consistency constraint, where we also enable to progressively estimate a more accurate surface. Specifically, we train a neural network to gradually infer the relationship between 3D queries and the approximated surface by searching for the moving target of queries in a dynamic way, which results in a consistent field around the surface. Meanwhile, we introduce a polygonization algorithm to extract surfaces directly from the gradient field of the learned UDF. The experimental results in surface reconstruction for synthetic and real scan data show significant improvements over the state-of-the-art under the widely used benchmarks. Project page: https://junshengzhou.github.io/CAP-UDF.

show abstract

Self-Supervised Point Cloud Representation Learning with Occlusion Auto-Encoder

Cited by 3 publications

References 0 publications

RSF: Optimizing Rigid Scene Flow From 3D Point Clouds Without Labels

RSF: Optimizing Rigid Scene Flow From 3D Point Clouds Without Labels

Upsampling Autoencoder for Self-Supervised Point Cloud Learning

Learning Consistency-Aware Unsigned Distance Functions Progressively from Raw Point Clouds

Contact Info

Product

Resources

About