A multi-view recurrent neural network for 3D mesh segmentation

Le, Tu; Bui, Giang; Duan, Ye

doi:10.1016/j.cag.2017.05.011

Cited by 87 publications

(38 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning methods, especially CNNs, have produced great achievements in image and speech processing tasks because of their powerful feature learning capability [4][5][6]. Recently, researchers have also attempted to generalize deep convolutional networks in regular grid domains to irregular 3D point clouds, which can be mainly summarized as voxelization-based [40,41], multi-view-based [16,18], graph-based [7,[42][43][44], and set-based [1,19] methods.…”

Section: Deep Learning On Point Cloudsmentioning

confidence: 99%

“…However, since the point clouds only record the surface points of the 3D objects, the voxelization-based method inevitably leads to resolution limitation, information loss, and computation consumption. The multi-view-based method [16][17][18] projects the 3D point clouds onto a series of 2D images from multiple views, so that the standard 2D CNNs can be applied directly. However, the multi-view-based method is occlusion sensitive, and it is still unclear how to determine the number, order, and distribution of the views to cover the entire 3D object while avoiding mutual occlusions.…”

Section: Deep Learning On Point Cloudsmentioning

confidence: 99%

“…However, because of the irregular data structure of 3D point clouds, standard convolutional neural networks (CNNs) cannot be directly applied to them. To address this problem, the most intuitive way is to divide the 3D point cloud space into regular 3D voxels [11][12][13][14] or project the 3D point cloud onto 2D images from multiple views [15][16][17][18], so that the CNNs can be applied directly. However, since the 3D point clouds only record the surface points of 3D objects, the 3D volumetric representation inevitably leads to computation consumption and resolution limitation, while the multi-view projection is sensitive to the mutual occlusion among objects [7].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation

et al. 2020

View full text Add to dashboard Cite

Semantic feature learning on 3D point clouds is quite challenging because of their irregular and unordered data structure. In this paper, we propose a novel structure-aware convolution (SAC) to generalize deep learning on regular grids to irregular 3D point clouds. Similar to the template-matching process of convolution on 2D images, the key of our SAC is to match the point clouds’ neighborhoods with a series of 3D kernels, where each kernel can be regarded as a “geometric template” formed by a set of learnable 3D points. Thus, the interested geometric structures of the input point clouds can be activated by the corresponding kernels. To verify the effectiveness of the proposed SAC, we embedded it into three recently developed point cloud deep learning networks (PointNet, PointNet++, and KCNet) as a lightweight module, and evaluated its performance on both classification and segmentation tasks. Experimental results show that, benefiting from the geometric structure learning capability of our SAC, all these back-end networks achieved better classification and segmentation performance (e.g., +2.77% mean accuracy for classification and +4.99% mean intersection over union (IoU) for segmentation) with few additional parameters. Furthermore, results also demonstrate that the proposed SAC is helpful in improving the robustness of networks with the constraints of geometric structures.

show abstract

Section: Deep Learning On Point Cloudsmentioning

confidence: 99%

Section: Deep Learning On Point Cloudsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation

et al. 2020

View full text Add to dashboard Cite

show abstract

“…3D shape analysis and component reuse require efficient semantic 3D shape segmentation. Tremendous progress in 3D shape segmentation has been made in the past decade . With the recent popularity of 3D printing, some segmentation approaches specifically designed for decomposing 3D models have been proposed.…”

Section: Related Workmentioning

confidence: 99%

Screwing assembly oriented interactive model segmentation in HMD VR environment

Zhu

Song

Wang

et al. 2019

Computer Animation & Virtual

View full text Add to dashboard Cite

Although different approaches of segmenting and assembling geometric models for 3D printing have been proposed, it is difficult to find any research studies, which investigate model segmentation and assembly in head‐mounted display (HMD) virtual reality (VR) environments for 3D printing. In this work, we propose a novel and interactive segmentation method for screwing assembly in the environments to tackle this problem. Our approach divides a large model into semantic parts with a screwing interface for repeated tight assembly. Specifically, after a user places the cutting interface, our algorithm computes the bounding box of the current part automatically for subsequent multicomponent semantic Boolean segmentations. Afterwards, the bolt is positioned with an improved K3M image thinning algorithm and is used for merging paired components with union and subtraction Boolean operations respectively. Moreover, we introduce a swept Boolean‐based rotation collision detection and location method to guarantee a collision‐free screwing assembly. Experiments show that our approach provides a new interactive multicomponent semantic segmentation tool that supports not only repeated installation and disassembly but also tight and aligned assembly.

show abstract

“…Current learning-based method can be divided into four categories: multiview-based, voxel-based, set-based and graphbased. Multiview-based and voxel-based methods (Le et al, 2017;Qi et al, 2016) represent the 3D shape into a set of images or regular volumetric occupancy grids, so that the feature learning method on regular arrays can be directly used. However, it is difficult to determine the distribution of the views, and the voxelbased method inevitably leads to memory and computation consumptions as they increase cubically with respect to the voxel's resolution.…”

Section: Related Workmentioning

confidence: 99%

Point Cloud Classification by Fusing Supervoxel Segmentation With Multi-Scale Features

Ao¹,

Wang²,

Shan³

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

<p><strong>Abstract.</strong> Point cloud classification is quite a challenging task due to the existence of noises, occlusion and various object types and sizes. Currently, the commonly used statistics-based features cannot accurately characterize the geometric information of a point cloud. This limitation often leads to feature confusion and classification mistakes (e.g., points of building corners and vegetation always share similar statistical features in a local neighbourhood, such as curvature, sphericity, etc). This study aims at solving this problem by leveraging the advantage of both the supervoxel segmentation and multi-scale features. For each point, its multi-scale features within different radii are extracted. Simultaneously, the point cloud is partitioned into simple supervoxel segments. After that, the class probability of each point is predicted by the proposed SegMSF approach that combines multi-scale features with the supervoxel segmentation results. At the end, the effect of data noises is supressed by using a global optimization that encourages spatial consistency of class labels. The proposed method is tested on both airborne laser scanning (ALS) and mobile laser scanning (MLS) point clouds. The experimental results demonstrate that the proposed method performs well in terms of classifying objects of different scales and is robust to noise.</p>

show abstract

A multi-view recurrent neural network for 3D mesh segmentation

Cited by 87 publications

References 43 publications

Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation

Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation

Screwing assembly oriented interactive model segmentation in HMD VR environment

Point Cloud Classification by Fusing Supervoxel Segmentation With Multi-Scale Features

Contact Info

Product

Resources

About