A Closer Look at Rotation-invariant Deep Point Cloud Analysis

Li, Feiran; Fujiwara, Kent; Okura, Fumio; Matsushita, Yasuyuki

doi:10.1109/iccv48922.2021.01591

Cited by 37 publications

(29 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our SO(3)-steerable model performs much better than its counterpart SE3CNN using comparable numbers of parameters, showing great expressive ability of our PDO-based filters. Compared with some recent point clouds based rotation equivariant models (KIM et al, 2020;Chen et al, 2021;Li et al, 2021), our method performs better, as volumetric data are more regular and can provide more structured and compact information. Also, our method shows great parameter efficiency (0.15M vs. 2.9M+).…”

Section: Shrec'17 Retrievalmentioning

confidence: 87%

PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

Shen¹,

Hong²,

She³

et al. 2022

Preprint

View full text Add to dashboard Cite

Steerable models can provide very general and flexible equivariance by formulating equivariance requirements in the language of representation theory and feature fields, which has been recognized to be effective for many vision tasks. However, deriving steerable models for 3D rotations is much more difficult than that in the 2D case, due to more complicated mathematics of 3D rotations. In this work, we employ partial differential operators (PDOs) to model 3D filters, and derive general steerable 3D CNNs, which are called PDO-s3DCNNs. We prove that the equivariant filters are subject to linear constraints, which can be solved efficiently under various conditions. As far as we know, PDO-s3DCNNs are the most general steerable CNNs for 3D rotations, in the sense that they cover all common subgroups of SO(3) and their representations, while existing methods can only be applied to specific groups and representations. Extensive experiments show that our models can preserve equivariance well in the discrete domain, and outperform previous works on SHREC'17 retrieval and ISBI 2012 segmentation tasks with a low network complexity. 1

show abstract

Section: Shrec'17 Retrievalmentioning

confidence: 87%

PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

Shen¹,

Hong²,

She³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, orientation may not be of interest when studying the shapes of human cells in suspension, or free-living marine organisms. In this study, we addressed this using pose correction techniques based on works in [60] (Figure 5). It is worth noting that pose correction caused a decrease in the extracted features' ability to distinguish between treatments.…”

Section: Discussionmentioning

confidence: 99%

3D single-cell shape analysis using geometric deep learning

Vries

Curry

Rowe-Brown

et al. 2022

Preprint

View full text Add to dashboard Cite

Aberrations in cell geometry are linked to cell signalling and disease. For example, metastatic melanoma cells alter their shape to invade tissues and drive disease. Despite this, there is a paucity of methods to quantify cell shape in 3D and little understanding of the shape-space cells explore. Currently, most descriptions of cell shape rely on predefined measurements of cell regions or points along a perimeter. The adoption of 3D tissue culture and imaging systems in medical research has recently created a growing need for comprehensive 3D shape descriptions of cells. We have addressed this need using unsupervised geometric deep learning to learn shape representations of cells from 3D microscopy images of metastatic melanoma cells embedded in collagen tissue-like matrices. We used a dynamic graph convolutional foldingnet autoencoder with improved deep embedded clustering to simultaneously learn lower-dimensional representations and classes of 3D cell shapes from a dataset of more than 70,000 drug-treated melanoma cells imaged by high throughput light-sheet microscopy. We propose describing cell shape using 3D quantitative morphological signatures, which represent a cell's similarity to shape modes in the dataset, and are a direct output from our model. We used the extracted features to reveal the extent of the cell shape landscape and found that the shapes learned could predict drug treatment (up to 86% accuracy) and cell microenvironment, and are explainable. In particular, we found strikingly similar deep learning shape signatures between cells treated with microtubule polymerisation inhibitors and branched actin inhibitors. Finally, we implemented our methods as a Python package for ease of use by the medical research community.

show abstract

“…However, it takes much space to cache the spherical harmonics calculated for irregular point clouds, leading to high space and time complexity. Concurrently, many works take efforts to design invariant operators [2,11,16,17,25,28,34], based on rotationally invariant measures such as distances and angles.…”

Section: Related Workmentioning

confidence: 99%

Equivariant Point Cloud Analysis via Learning Orientations for Message Passing

Luo¹,

Li²,

Guan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Equivariance has been a long-standing concern in various fields ranging from computer vision to physical modeling. Most previous methods struggle with generality, simplicity, and expressiveness -some are designed ad hoc for specific data types, some are too complex to be accessible, and some sacrifice flexible transformations. In this work, we propose a novel and simple framework to achieve equivariance for point cloud analysis based on the message passing (graph neural network) scheme. We find the equivariant property could be obtained by introducing an orientation for each point to decouple the relative position for each point from the global pose of the entire point cloud. Therefore, we extend current message passing networks with a module that learns orientations for each point. Before aggregating information from the neighbors of a point, the networks transforms the neighbors' coordinates based on the point's learned orientations. We provide formal proofs to show the equivariance of the proposed framework. Empirically, we demonstrate that our proposed method is competitive on both point cloud analysis and physical modeling tasks. Code is available at https://github.com/ luost26/Equivariant-OrientedMP.

show abstract

A Closer Look at Rotation-invariant Deep Point Cloud Analysis

Cited by 37 publications

References 22 publications

PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

3D single-cell shape analysis using geometric deep learning

Equivariant Point Cloud Analysis via Learning Orientations for Message Passing

Contact Info

Product

Resources

About