Learning SO(3) Equivariant Representations with Spherical CNNs

Esteves, Carlos; Allen-Blanchette, Christine; Makadia, Ameesh; Daniilidis, Kostas

doi:10.1007/978-3-030-01261-8_4

Cited by 238 publications

(54 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is highly relevant when one seeks to solve inverse problems whose solutions enjoy such equivariance and/or invariance. Examples of work in this direction are those of Esteves, Allen-Blanchette, Makadia and Daniilidis (2017), Zhao et al. (2018), Weiler et al.…”

Section: Discussionmentioning

confidence: 99%

Solving inverse problems using data-driven models

et al. 2019

View full text Add to dashboard Cite

Recent research in inverse problems seeks to develop a mathematically coherent foundation for combining data-driven models, and in particular those based on deep learning, with domain-specific knowledge contained in physical–analytical models. The focus is on solving ill-posed inverse problems that are at the core of many challenging applications in the natural sciences, medicine and life sciences, as well as in engineering and industrial applications. This survey paper aims to give an account of some of the main contributions in data-driven inverse problems.

show abstract

Section: Discussionmentioning

confidence: 99%

Solving inverse problems using data-driven models

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The transformed image is then segmented using a convolutional neural network (CNN). Esteves et al [9] proposed a polar transformer network for image classification. Note that "transformer network" here refers to spatial transformer networks [10] and not attention-based networks commonly called transformers.…”

Section: A Related Work 1) Combining Polar Coordinates and Neural Networkmentioning

confidence: 99%

“…Finally, to increase the model's robustness to suboptimal center point predictions, we augment the calculated center for training images [9]. Each training image has a 30% chance of varying the center's x and y coordinates by a random value in the range (−S • 0.05, S • 0.05), where S is the smallest resolution of the image, i.e.…”

Section: B Training a Network On Polar Imagesmentioning

confidence: 99%

Training on Polar Image Transformations Improves Biomedical Image Segmentation

et al. 2021

View full text Add to dashboard Cite

A key step in medical image-based diagnosis is image segmentation. A common use case for medical image segmentation is the identification of single structures of an elliptical shape. Most organs like the heart and kidneys fall into this category, as well as skin lesions, polyps, and other types of abnormalities. Neural networks have dramatically improved medical image segmentation results, but still require large amounts of training data and long training times to converge. In this paper, we propose a general way to improve neural network segmentation performance and data efficiency on medical imaging segmentation tasks where the goal is to segment a single roughly elliptically distributed object. We propose training a neural network on polar transformations of the original dataset, such that the polar origin for the transformation is the center point of the object. This results in a reduction of dimensionality as well as a separation of segmentation and localization tasks, allowing the network to more easily converge. Additionally, we propose two different approaches to obtaining an optimal polar origin: (1) estimation via a segmentation trained on non-polar images and (2) estimation via a model trained to predict the optimal origin. We evaluate our method on the tasks of liver, polyp, skin lesion, and epicardial adipose tissue segmentation. We show that our method produces state-of-the-art results for lesion, liver, and polyp segmentation and performs better than most common neural network architectures for biomedical image segmentation. Additionally, when used as a pre-processing step, our method generally improves data efficiency across datasets and neural network architectures.

show abstract

“…Esteves at al. presented a Polar Transformer Network (PTN) model to classify objects with rotational invariance, also extending the model to 3D objects with the use of a cylindrical coordinate system [38].…”

Section: Et Al Used Scale Invariant Convolutionalmentioning

confidence: 99%

Spherical coordinates transformation pre-processing in Deep Convolution Neural Networks for brain tumor segmentation in MRI

Russo

Liu

Ieva

2021

Med Biol Eng Comput

View full text Add to dashboard Cite

Background and Objective: Magnetic Resonance Imaging (MRI) is used in everyday clinical practice to assess brain tumors. Several automatic or semi-automatic segmentation algorithms have been introduced to segment brain tumors and achieve an expert-like accuracy. Deep Convolutional Neural Networks (DCNN) have recently shown very promising results, however, DCNN models are still far from achieving clinically meaningful results mainly because of the lack of generalization of the models. DCNN models need large annotated datasets to achieve good performance. Models are often optimized on the domain dataset on which they have been trained, and then fail the task when the same model is applied to different datasets from different institutions. One of the reasons is due to the lack of data standardization to adjust for different models and MR machines. Methods:In this work, a 3D Spherical coordinates transform during the pre-processing phase has been hypothesized to improve DCNN models' accuracy and to allow more generalizable results even when the model is trained on small and heterogeneous datasets and translated into different domains. Indeed, the spherical coordinate system avoids several standardization issues since it works independently of resolution and imaging settings. Both Cartesian and spherical volumes were evaluated in two DCNN models with the same network structure using the BraTS 2019 dataset. Results:The model trained on spherical transform pre-processed inputs resulted in superior performance over the Cartesian-input trained model on predicting gliomas' segmentation on tumor core and enhancing tumor classes (increase of 0.011 and 0.014 respectively on the validation dataset), achieving a further improvement in accuracy by merging the two models together. Furthermore, the spherical transform is not resolution-dependent and achieve same results on different input resolution. Conclusions:Our model is achieving competitive dice scores and is not resolution-dependent, therefore can be applied to different datasets without re-training, thus improving segmentation accuracy and theoretically solving some transfer learning problems related to the domain shifting, at least in terms of image resolution in the datasets.

show abstract

Learning SO(3) Equivariant Representations with Spherical CNNs

Cited by 238 publications

References 46 publications

Solving inverse problems using data-driven models

Solving inverse problems using data-driven models

Training on Polar Image Transformations Improves Biomedical Image Segmentation

Spherical coordinates transformation pre-processing in Deep Convolution Neural Networks for brain tumor segmentation in MRI

Contact Info

Product

Resources

About