Parallel Transport Convolution: Deformable Convolutional Networks on Manifold-Structured Data

Schonsheck, Stefan C.; Dong, Bin; Lai, Rongjie

doi:10.1137/21m1407616

Cited by 6 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the key challenges to applying wavelets and similar constructions to nodebased graph signals is that graphs lack a natural translation operator, which prevents the construction of convolutional operators and traditional Littlewood-Paley theory [19,25,44]. This challenge is also present for general κ-dimensional simplices.…”

Section: Related Workmentioning

confidence: 99%

“…Unfortunately, these definitions do not generalize to non-homogeneous domains due to the lack of appropriate translation operators and dilation operators [44]. Instead, several methods have been proposed to generate similar bases, and overcomplete dictionaries to apply more abstract domains such as graphs and discretized manifolds [17,45,40].…”

Section: Orthonormal κ-Haarmentioning

confidence: 99%

See 1 more Smart Citation

Multiscale Transforms for Signals on Simplicial Complexes

Saito¹,

Schonsheck²,

Shvarts³

2023

Preprint

View full text Add to dashboard Cite

Our previous multiscale graph basis dictionaries/graph signal transforms-Generalized Haar-Walsh Transform (GHWT); Hierarchical Graph Laplacian Eigen Transform (HGLET); Natural Graph Wavelet Packets (NGWPs); and their relatives-were developed for analyzing data recorded on nodes of a given graph. In this article, we propose their generalization for analyzing data recorded on edges, faces (i.e., triangles), or more generally κ-dimensional simplices of a simplicial complex (e.g., a triangle mesh of a manifold). The key idea is to use the Hodge Laplacians and their variants for hierarchical partitioning of a set of κ-dimensional simplices in a given simplicial complex, and then build localized basis functions on these partitioned subsets. We demonstrate their usefulness for data representation on both illustrative synthetic examples and real-world simplicial complexes generated from a co-authorship/citation dataset and an ocean current/flow dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Orthonormal κ-Haarmentioning

confidence: 99%

Multiscale Transforms for Signals on Simplicial Complexes

Saito¹,

Schonsheck²,

Shvarts³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition, defining the localized convolution kernels is necessary for this representation; strategies include localizing spectral filters [28], utilizing the classic geometric approaches (e.g., B-splines [29], wavelets [30], and extrinsic Euclidean convolution [31]). The methods with local parameterization also introduce a drawback that only the small receptive field is applied to aggregate the information from the surface meshes, wherefore the performances of these approaches depend on the mesh resolution of the 3D objects.…”

Section: Geometric Deep Learning On Meshesmentioning

confidence: 99%

Laplacian2Mesh: Laplacian-Based Mesh Understanding

Qian¹,

Wang²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Geometric deep learning has sparked a rising interest in computer graphics to perform shape understanding tasks, such as shape classification and semantic segmentation on three-dimensional (3D) geometric surfaces. Previous works explored the significant direction by defining the operations of convolution and pooling on triangle meshes, but most methods explicitly utilized the graph connection structure of the mesh. Motivated by the geometric spectral surface reconstruction theory, we introduce a novel and flexible convolutional neural network (CNN) model, called Laplacian2Mesh, for 3D triangle mesh, which maps the features of mesh in the Euclidean space to the multi-dimensional Laplacian-Beltrami space, which is similar to the multi-resolution input in 2D CNN. Mesh pooling is applied to expand the receptive field of the network by the multi-space transformation of Laplacian which retains the surface topology, and channel self-attention convolutions are applied in the new space. Since implicitly using the intrinsic geodesic connections of the mesh through the adjacency matrix, we do not consider the number of the neighbors of the vertices, thereby mesh data with different numbers of vertices can be input. Experiments on various learning tasks applied to 3D meshes demonstrate the effectiveness and efficiency of Laplacian2Mesh.

show abstract

“…In this work, we are interested in rotationally invariant features, thus we take a path closer to Schonsheck et al 17 and Masci et al, 7 we actually lift functions to functions on the bundle of tangent space rotations of our manifolds, a two-dimensional manifold, as opposed to Cohen et al 18 where the lifting results in functions on SOð3Þ-a three-dimensional manifold. Then, we add one or more extra local group convolution layers before summarising the data and eliminating path dependency.…”

Section: Introductionmentioning

confidence: 99%

“…[9][10][11][12][13][14][15] Global equivariance is often sought but proved complicated or even elusive in many cases when the underlying geometry is nontrivial. 16 An elementary construction on a general manifold is proposed by Schonsheck et al 17 via a fixed choice of geodesic paths used to transport filters between points on the manifold, ignoring the effects of path dependency (holonomy when paths are geodesics). The removal of this dependency can be obtained by summarizing local responses over local orientations, which is what was done by Masci et al 7 On the other hand, Cohen et al 18 lifted spherical functions to the 3D-rotation group SOð3Þ and used a generalization of Fourier transform on it to perform convolution.…”

Section: Introductionmentioning

confidence: 99%

Bundle geodesic convolutional neural network for diffusion-weighted imaging segmentation

et al. 2022

View full text Add to dashboard Cite

Purpose: Applying machine learning techniques to magnetic resonance diffusion-weighted imaging (DWI) data is challenging due to the size of individual data samples and the lack of labeled data. It is possible, though, to learn general patterns from a very limited amount of training data if we take advantage of the geometry of the DWI data. Therefore, we present a tissue classifier based on a Riemannian deep learning framework for single-shell DWI data.Approach: The framework consists of three layers: a lifting layer that locally represents and convolves data on tangent spaces to produce a family of functions defined on the rotation groups of the tangent spaces, i.e., a (not necessarily continuous) function on a bundle of rotational functions on the manifold; a group convolution layer that convolves this function with rotation kernels to produce a family of local functions over each of the rotation groups; a projection layer using maximization to collapse this local data to form manifold based functions.Results: Experiments show that our method achieves the performance of the same level as stateof-the-art while using way fewer parameters in the model (<10%). Meanwhile, we conducted a model sensitivity analysis for our method. We ran experiments using a proportion (69.2%, 53.3%, and 29.4%) of the original training set and analyzed how much data the model needs for the task. Results show that this does reduce the overall classification accuracy mildly, but it also boosts the accuracy for minority classes.Conclusions: This work extended convolutional neural networks to Riemannian manifolds, and it shows the potential in understanding structural patterns in the brain, as well as in aiding manual data annotation.

show abstract

Parallel Transport Convolution: Deformable Convolutional Networks on Manifold-Structured Data

Cited by 6 publications

References 11 publications

Multiscale Transforms for Signals on Simplicial Complexes

Multiscale Transforms for Signals on Simplicial Complexes

Laplacian2Mesh: Laplacian-Based Mesh Understanding

Bundle geodesic convolutional neural network for diffusion-weighted imaging segmentation

Contact Info

Product

Resources

About