Parsing Geometry Using Structure-Aware Shape Templates

Abstract

Real-life man-made objects often exhibit strong and easily-identifiable structure, as a direct result of their design or their intended functionality. Structure typically appears in the form of individual parts and their arrangement. Knowing about object structure can be an important cue for object recognition and scene understanding -a key goal for various AR and robotics applications. However, commodity RGB-D sensors used in these scenarios only produce raw, unorganized point clouds, without structural information about the captured scene. Moreover, the generated data is commonly partial and susceptible to artifacts and noise, which makes inferring the structure of scanned objects challenging. In this paper, we organize large shape collections into parameterized shape templates to capture the underlying structure of the objects. The templates allow us to transfer the structural information onto new objects and incomplete scans. We employ a deep neural network that matches the partial scan with one of the shape templates, then match and fit it to complete and detailed models from the collection. This allows us to faithfully label its parts and to guide the reconstruction of the scanned object. We showcase the effectiveness of our method by comparing it to other state-of-the-art approaches.

Supervised Fitting of Geometric Primitives to 3D Point Clouds

“…Representing an object with a set of simple geometric components is a long-standing problem in computer vision. Since the 1970s [3,19], the fundamental ideas for tackling the problem have been revised by many researchers, even until recently [32,35,9]. However, most of these previous work aimed at solving perceptual learning tasks; the main focus was on parsing shapes, or generating a rough abstraction of the geometry with bounding primitives.…”

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

“…There is a line of research on understanding shapes by their semantic parts and structures. Previous works study 3D shape part segmentation [7,36,91,34,55,92,77,51,10], generate shape box abstraction [71,99,53,64], shape template fitting [40,17,19], generate shapes by explicitly modeling parts and structures [35,43,65,76,82,69,49,81,18,60,42], or edit shape by parts [12,95,50].…”

Learning to Infer Semantic Parameters for 3D Shape Editing

“…Their shape decomposition works for various man-made shapes, but is mostly based on geometry and is not semantically consistent across shapes. To achieve semantic consistency, Ganapathi-Subramanian et al [4] fit each input shape with a class-specific refined template. However, such fitting algorithms are fragile, utilizing hand-crafted heuristics and thresholds.…”

“…Traditional shape manipulation methods [4,27] first fit predefined handles (e.g., cages, primitives) to input shapes through optimization. The user then manipulates the tem-Figure 1: Semantic editing.…”