DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion

Abstract

Point cloud processing is a challenging task due to its sparsity and irregularity. Prior works introduce delicate designs on either local feature aggregator or global geometric architecture, but few combine both advantages. We propose Dual-Scale Point Cloud Recognition with Highfrequency Fusion (DSPoint) to extract local-global features by concurrently operating on voxels and points. We reverse the conventional design of applying convolution on voxels and attention to points. Specifically, we disentangle point features through channel dimension for dual-scale processing: one by point-wise convolution for fine-grained geometry parsing, the other by voxel-wise global attention for long-range structural exploration. We design a co-attention fusion module for feature alignment to blend local-global modalities, which conducts inter-scale cross-modality interaction by communicating high-frequency coordinates information. Experiments and ablations on widely-adopted ModelNet40, ShapeNet, and S3DIS demonstrate the stateof-the-art performance of our DSPoint. Our code is available at: https://github.com/Adonis-galaxy/ DSPoint.

A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation

“…In DSPoint [79], Zhang et al introduced a dual-scale point cloud recognition approach that combines local features and global geometric architecture. Unlike conventional designs, DSPoint operates concurrently on voxels and points, extracting local and global features.…”

Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation

“…Prototypes (i.e., class centroids) are non-learnable vectors in the feature space that are representative of each semantic category that appears in the dataset [44], [72], [80]. During training, the features extracted by the encoder contribute in forming the latent prototypical representation both for micro and macro classes.…”