Global Context Aware Convolutions for 3D Point Cloud Understanding

Zhang, Z.; Hua, Binh-Son; Chen, Wei; Tian, Yulu; Yeung, Sai-Kit

doi:10.1109/3dv50981.2020.00031

Cited by 31 publications

(27 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One way is use layers that produce rotationally invariant features, which can then be processed further without restrictions, for example by aligning the inputs to the convolutional filters. This approach is taken by GCANet [37], and by MA-KPConv [28], which extends the KPConv model by using multiple alignments for the filters. Another approach is to apply an invariant map as the final function of each layer, as in Spherical Harmonics Networks (SPHnet) [22], which calculate activations in a spherical harmonic basis and produce invariant output by taking the norm over coefficients with identical degree.…”

Section: Related Workmentioning

confidence: 99%

Nonlinearities in Steerable SO(2)-Equivariant CNNs

Franzen,

Wand

2021

Preprint

View full text Add to dashboard Cite

Invariance under symmetry is an important problem in machine learning. Our paper looks specifically at equivariant neural networks where transformations of inputs yield homomorphic transformations of outputs. Here, steerable CNNs have emerged as the standard solution. An inherent problem of steerable representations is that general nonlinear layers break equivariance, thus restricting architectural choices. Our paper applies harmonic distortion analysis to illuminate the effect of nonlinearities on Fourier representations of SO(2). We develop a novel FFT-based algorithm for computing representations of non-linearly transformed activations while maintaining band-limitation. It yields exact equivariance for polynomial (approximations of) nonlinearities, as well as approximate solutions with tunable accuracy for general functions. We apply the approach to build a fully E(3)equivariant network for sampled 3D surface data. In experiments with 2D and 3D data, we obtain results that compare favorably to the state-of-the-art in terms of accuracy while permitting continuous symmetry and exact equivariance.G-CNNs: To make current CNN architectures (consisting of linear layers and nonlinearities) equivariant, the standard (and also most general, see [32]) approach are group convolutional networks Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Nonlinearities in Steerable SO(2)-Equivariant CNNs

Franzen,

Wand

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Particularly, we propose an effective and lightweight approach to perform rotation-invariant convolution for point clouds, which is an extension of our previous work (Zhang et al 2019a(Zhang et al , 2020. We propose to make rotation-invariant features more informative by local reference axis (LRA), and consider point-point relations, which improves feature distinction as well.…”

Section: Introductionmentioning

confidence: 97%

“…However, in 3D, such data augmentation becomes less effective due to the additional degrees of freedom, which can make training prohibitively expensive. Some previous works have been proposed to learn rotation-invariant features (Zhang et al 2020(Zhang et al , 2019aRao et al 2019;Poulenard et al 2019;Deng et al 2018;Chen et al 2019), which leads to consistent predictions given arbitrarily rotated point clouds. We observe that state-of-the-art methods can improve the feature learning by using local reference frame (LRF) to encode both local and global information (Zhang et al 2020;Kim et al 2020b;Thomas 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Some previous works have been proposed to learn rotation-invariant features (Zhang et al 2020(Zhang et al , 2019aRao et al 2019;Poulenard et al 2019;Deng et al 2018;Chen et al 2019), which leads to consistent predictions given arbitrarily rotated point clouds. We observe that state-of-the-art methods can improve the feature learning by using local reference frame (LRF) to encode both local and global information (Zhang et al 2020;Kim et al 2020b;Thomas 2020). However, LRF usually suffers sign flipping problem in the x and y axes, which makes rotation-invariant convolutions built upon LRF yield features not as distinctive as a translationinvariant convolution does.…”

Section: Introductionmentioning

confidence: 99%

“…This can be demonstrated in the result of the object classification task: performing classification with aligned 3D shapes (using translationinvariant convolution) is more accurate than performing the same task with shapes with arbitrary rotations (using rotation-invariant convolution). For example, stateof-the-art methods with rotation invariance (Zhang et al 2019a;Poulenard et al 2019;Zhang et al 2020;Kim et al 2020b) reported classification accuracies about 86%-89% on ModelNet40 Wu et al (2015a) while methods without rotation invariance can reach accuracy as high as 93% (Wang et al 2019;Zhang et al 2019b). This motivates us to address the limitation of the LRF and design a convolution that gives more informative features to increase the overall performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

RIConv++: Effective Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

Zhang¹,

Hua²,

Yeung³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks. While recent works show that point cloud convolutions can be invariant to translation and point permutation, investigations of the rotation invariance property for point cloud convolution has been so far scarce. Some existing methods perform point cloud convolutions with rotation-invariant features, existing methods generally do not perform as well as translation-invariant only counterpart. In this work, we argue that a key reason is that compared to point coordinates, rotation-invariant features consumed by point cloud convolution are not as distinctive. To address this problem, we propose a simple yet effective convolution operator that enhances feature distinction by designing powerful rotation invariant features from the local regions. We consider the relationship between the point of interest and its neighbors as well as the internal relationship of the neighbors to largely improve the feature descriptiveness. Our network architecture can capture both local and global context by simply tuning the neighborhood size in each convolution layer. We conduct several experiments on synthetic and real-world

show abstract