Progressive Seed Generation Auto-encoder for Unsupervised Point Cloud Learning

Yang, Juyoung; Ahn, Pyunghwan; Kim, Doyeon; Lee, Haeil; Kim, Junmo

doi:10.1109/iccv48922.2021.00635

Cited by 20 publications

(15 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many studies have used deep learning to cluster or organise everyday 3D objects by their shape [31, 44, 45, 46]. These tasks use datasets that have distinct semantic (or human meaningful) classes.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

3D single-cell shape analysis using geometric deep learning

Vries

Curry

Rowe-Brown

et al. 2022

Preprint

View full text Add to dashboard Cite

Aberrations in cell geometry are linked to cell signalling and disease. For example, metastatic melanoma cells alter their shape to invade tissues and drive disease. Despite this, there is a paucity of methods to quantify cell shape in 3D and little understanding of the shape-space cells explore. Currently, most descriptions of cell shape rely on predefined measurements of cell regions or points along a perimeter. The adoption of 3D tissue culture and imaging systems in medical research has recently created a growing need for comprehensive 3D shape descriptions of cells. We have addressed this need using unsupervised geometric deep learning to learn shape representations of cells from 3D microscopy images of metastatic melanoma cells embedded in collagen tissue-like matrices. We used a dynamic graph convolutional foldingnet autoencoder with improved deep embedded clustering to simultaneously learn lower-dimensional representations and classes of 3D cell shapes from a dataset of more than 70,000 drug-treated melanoma cells imaged by high throughput light-sheet microscopy. We propose describing cell shape using 3D quantitative morphological signatures, which represent a cell's similarity to shape modes in the dataset, and are a direct output from our model. We used the extracted features to reveal the extent of the cell shape landscape and found that the shapes learned could predict drug treatment (up to 86% accuracy) and cell microenvironment, and are explainable. In particular, we found strikingly similar deep learning shape signatures between cells treated with microtubule polymerisation inhibitors and branched actin inhibitors. Finally, we implemented our methods as a Python package for ease of use by the medical research community.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Following similar procedures to previous researchers [31,44,45,46], we used the trained DFN to extract features from each cell. These features were then used to train a linear SVM.…”

Section: Learned Shape Features Predict Cell Treatmentmentioning

confidence: 99%

3D single-cell shape analysis using geometric deep learning

Vries

Curry

Rowe-Brown

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Currently, self-supervised learning has been extensively studied in point cloud representation learning, and they focus on constructing a pretext task to help the network better learn 3D point cloud representations. A commonly adopted pretext task is to reconstruct the input point cloud from the latent encoding space, which can be implemented through Variational AutoEncoders [29][30][31][32][33][34], Generative Adversarial Learning (GANs) [35,36], Gaussian Mixed Model [12,37], etc. However, these methods are computationally expensive, and rely excessively on reconstructing local details, making it difficult to learn high-level semantic features.…”

Section: Related Workmentioning

confidence: 99%

POS-BERT: Point Cloud One-Stage BERT Pre-Training

Fu¹,

Gao²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently, the pre-training paradigm combining Transformer and masked language modeling has achieved tremendous success in NLP, images, and point clouds, such as BERT. However, directly extending BERT from NLP to point clouds requires training a fixed discrete Variational AutoEncoder (dVAE) before pre-training, which results in a complex two-stage method called Point-BERT. Inspired by BERT and MoCo, we propose POS-BERT, a one-stage BERT pre-training method for point clouds. Specifically, we use the mask patch modeling (MPM) task to perform point cloud pre-training, which aims to recover masked patches information under the supervision of the corresponding tokenizer output. Unlike Point-BERT, its tokenizer is extra-trained and frozen. We propose to use the dynamically updated momentum encoder as the tokenizer, which is updated and outputs the dynamic supervision signal along with the training process. Further, in order to learn high-level semantic representation, we combine contrastive learning to maximize the class token consistency between different transformation point clouds. Extensive experiments have demonstrated that POS-BERT can extract high-quality pre-training features and promote downstream tasks to improve performance. Using the pretraining model without any fine-tuning to extract features and train linear SVM on ModelNet40, POS-BERT achieves the state-of-the-art classification accuracy, which exceeds Point-BERT by 3.5%. In addition, our approach has significantly improved many downstream tasks, such as fine-tuned classification, few-shot classification, part segmentation. The code and trained-models will be available at: https://github.com/fukexue/POS-BERT.

show abstract

“…For instance, the encoder of 3DCapsuleNet [49] is designed to process 3D point clouds and aggregate the final latent capsules based on the dynamic routing, followed by the decoder to reconstruct the original point cloud composed of multiple point patches. [45] proposes an autoencoder architecture where the rich point-wise features in multiple stages are progressively produced through the seed generation module. Eckart et al [13] further extend the traditional autoencoder paradigm to form the bottleneck layer being modeled by a discrete generative model.…”

Section: Related Workmentioning

confidence: 99%

“…GraphTER † [16] 87.8 FoldingNet [46] 88.4 PointCapsNet [49] 88.9 Multi-Tasks [19] 89.1 Yang et al [45] 90.9 PointGLR † [34] 91.7…”

Section: Ssl Accuracymentioning

confidence: 99%

Self-Supervised Feature Learning from Partial Point Clouds via Pose Disentanglement

Tsai¹,

Chiang²,

Tsai³

et al. 2022

Preprint

View full text Add to dashboard Cite

Self-supervised learning on point clouds has gained a lot of attention recently, since it addresses the label-efficiency and domain-gap problems on point cloud tasks. In this paper, we propose a novel self-supervised framework to learn informative representations from partial point clouds. We leverage partial point clouds scanned by LiDAR that contain both content and pose attributes, and we show that disentangling such two factors from partial point clouds enhances feature representation learning. To this end, our framework consists of three main parts: 1) a completion network to capture holistic semantics of point clouds; 2) a pose regression network to understand the viewing angle where partial data is scanned from; 3) a partial reconstruction network to encourage the model to learn content and pose features. To demonstrate the robustness of the learnt feature representations, we conduct several downstream tasks including classification, part segmentation, and registration, with comparisons against state-of-the-art methods. Our method not only outperforms existing self-supervised methods, but also shows a better generalizability across synthetic and realworld datasets.

show abstract

Progressive Seed Generation Auto-encoder for Unsupervised Point Cloud Learning

Cited by 20 publications

References 12 publications

3D single-cell shape analysis using geometric deep learning

3D single-cell shape analysis using geometric deep learning

POS-BERT: Point Cloud One-Stage BERT Pre-Training

Self-Supervised Feature Learning from Partial Point Clouds via Pose Disentanglement

Contact Info

Product

Resources

About