Data-driven shape analysis and processing

Xu, Kai; Kim, Vladimir G.; Huang, Qixing; Mitra, Niloy J.; Kalogerakis, Evangelos

doi:10.1145/2988458.2988473

Cited by 90 publications

(68 citation statements)

References 390 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent shape analysis techniques focus on extracting structure from large collections of 3D models [Xu et al 2016]. In this section we discuss recent work on detecting labeled parts and hierarchies in shape collections.…”

Section: Related Workmentioning

confidence: 99%

Learning hierarchical shape segmentation and labeling from online repositories

Guibas

Hertzmann

et al. 2017

ACM Trans. Graph.

Self Cite

View full text Add to dashboard Cite

Figure 1: Large online model repositories contain abundant additional data beyond 3D geometry, such as part labels and artist's part decompositions, flat or hierarchical. We tap into this trove of sparse and noisy noisy data to train a network for simultaneous hierarchical shape structure decomposition and labeling. Our method learns to take new geometry, and segment it into parts, label the parts, and place them in a hierarchy. In this paper, we visualize scene graphs with a circular visualization, in which the root node is near the center. Blue lines indicate parent-child relationships, and red dashed arcs connect siblings. The input geometry in online databases are broken as connected components, visualized in the input as random colors. AbstractWe propose a method for converting geometric shapes into hierarchically segmented parts with part labels. Our key idea is to train category-specific models from the scene graphs and part names that accompany 3D shapes in public repositories. These freelyavailable annotations represent an enormous, untapped source of information on geometry. However, because the models and corresponding scene graphs are created by a wide range of modelers with different levels of expertise, modeling tools, and objectives, these models have very inconsistent segmentations and hierarchies with sparse and noisy textual tags. Our method involves two analysis steps. First, we perform a joint optimization to simultaneously cluster and label parts in the database while also inferring a canonical tag dictionary and part hierarchy. We then use this labeled data to train a method for hierarchical segmentation and labeling of new 3D shapes. We demonstrate that our method can mine complex information, detecting hierarchies in man-made objects and their constituent parts, obtaining finer scale details than existing alternatives. We also show that, by performing domain transfer using a few supervised examples, our technique outperforms fully-supervised techniques that require hundreds of manually-labeled models.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning hierarchical shape segmentation and labeling from online repositories

Guibas

Hertzmann

et al. 2017

ACM Trans. Graph.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Data‐Driven Shape Processing Xu et al [XKHK17] survey recent data‐driven methods for shape processing including part based modeling [FKS∗04, KJS07], sketch‐based modeling [FWX∗13, XXM∗13] and shape editing [XZZ∗11, ZCOM13]. Similar to us, these methods attempt to simplify 3D modeling by leveraging existing data.…”

Section: Related Workmentioning

confidence: 99%

String‐Based Synthesis of Structured Shapes

Kalojanov

Lim

Mitra³

et al. 2019

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

We propose a novel method to synthesize geometric models from a given class of context‐aware structured shapes such as buildings and other man‐made objects. The central idea is to leverage powerful machine learning methods from the area of natural language processing for this task. To this end, we propose a technique that maps shapes to strings and vice versa, through an intermediate shape graph representation. We then convert procedurally generated shape repositories into text databases that, in turn, can be used to train a variational autoencoder. The autoencoder enables higher level shape manipulation and synthesis like, for example, interpolation and sampling via its continuous latent space. We provide project code and pre‐trained models.

show abstract

“…A wide range of fundamental shape analysis problems such as classification [BBGO11], segmentation [KHS10], and correspondence [COC14] have been addressed with machine learning techniques (see [XKH*16] for a survey). Due to recent developments and success of deep neural networks, researchers have focused on developing appropriate shape representations suitable for deep learning.…”

Section: Related Workmentioning

confidence: 99%

GWCNN: A Metric Alignment Layer for Deep Shape Analysis

Ezuz

Solomon

Kim

et al. 2017

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Deep neural networks provide a promising tool for incorporating semantic information in geometry processing applications. Unlike image and video processing, however, geometry processing requires handling unstructured geometric data, and thus data representation becomes an important challenge in this framework. Existing approaches tackle this challenge by converting point clouds, meshes, or polygon soups into regular representations using, e.g., multi‐view images, volumetric grids or planar parameterizations. In each of these cases, geometric data representation is treated as a fixed pre‐process that is largely disconnected from the machine learning tool. In contrast, we propose to optimize for the geometric representation during the network learning process using a novel metric alignment layer. Our approach maps unstructured geometric data to a regular domain by minimizing the metric distortion of the map using the regularized Gromov–Wasserstein objective. This objective is parameterized by the metric of the target domain and is differentiable; thus, it can be easily incorporated into a deep network framework. Furthermore, the objective aims to align the metrics of the input and output domains, promoting consistent output for similar shapes. We show the effectiveness of our layer within a deep network trained for shape classification, demonstrating state‐of‐the‐art performance for nonrigid shapes.

show abstract

Data-driven shape analysis and processing

Abstract: Data-driven methods play an increasingly important role in discovering geometric

Cited by 90 publications

References 390 publications

Learning hierarchical shape segmentation and labeling from online repositories

Learning hierarchical shape segmentation and labeling from online repositories

String‐Based Synthesis of Structured Shapes

GWCNN: A Metric Alignment Layer for Deep Shape Analysis

Contact Info

Product

Resources

About