VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapes

Abstract

Voxel is an important format to represent geometric data, which has been widely used for 3D deep learning in shape analysis due to its generalization ability and regular data format. However, fine-grained tasks like part segmentation require detailed structural information, which increases voxel resolution and thus causes other issues such as the exhaustion of computational resources. In this paper, we propose a novel volumetric convolutional neural network, which could extract discriminative features encoding detailed information from voxelized 3D data under a limited resolution. To this purpose, a spatial dense extraction (SDE) module is designed to preserve the spatial resolution during the feature extraction procedure, alleviating the loss of detail caused by sub-sampling operations such as max-pooling. An attention feature aggregation (AFA) module is also introduced to adaptively select informative features from different abstraction scales, leading to segmentation with both semantic consistency and high accuracy of details. Experiment results on the large-scale dataset demonstrate the effectiveness of our method in 3D shape part segmentation.

PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding

“…Recent advances of 3D semantic segmentation [29,30,44,17,33,22,7,37,38,40,31,6,24,21] have accomplished promising achievement in coarse-level segmentation on the ShapeNet Part dataset [3,43]. However, few work focus on the fine-grained 3D semantic segmentation, due to the lack of large-scale fine-grained dataset.…”

“…With the availability of the existing 3D shape datasets with part annotations [5,3,43], we witness increasing research interests and advances in 3D part-level object understanding. Recently, a variety of learning methods have been proposed to push the state-of-the-art for 3D shape segmentation [29,30,44,17,33,22,7,37,38,40,31,6,24,21]. However, existing datasets only provide part annotations on relatively small numbers of object instances [5], or on coarse yet non-hierarchical part annotations [43], restricting the applications that involves understanding fine-grained and hierarchical shape segmentation.…”

“…There are many other algorithm candidates:[44,17,33,22,7,37,38,31,6,21]. We will host an online leadboard to report the performances.…”