PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

Mo, Kaichun; Wang, He; Yan, Xinchen; Guibas, Leonidas J.

doi:10.48550/arxiv.2003.08624

Cited by 4 publications

(5 citation statements)

References 100 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many previous works [8,35,31,23] learn to create a novel shape from scratch by embedding both the geometric and structural diversity into the generative networks. However, provided the part geometry, our problem only allows structural diversity to be modeled.…”

Section: E Additional Results Of Structural Variationmentioning

confidence: 99%

“…[31] presents a global-tolocal adversarial network to construct the overall structure of the shape, followed by a conditional autoencoder for part refinement. Recently, [23] employs a conditional GAN to generate a point cloud from an input rough shape structure. Most of the aforementioned works couple the shape structure and geometry into the joint learning for diverse and perceptually plausible 3D modeling.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Generative 3D Part Assembly via Dynamic Graph Learning

Huang,

Zhan,

Fan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous part assembly is a challenging yet crucial task in 3D computer vision and robotics. Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation. In this paper, we focus on the pose estimation subproblem from the vision side involving geometric and relational reasoning over the input part geometry. Essentially, the task of generative 3D part assembly is to predict a 6-DoF part pose, including a rigid rotation and translation, for each input part that assembles a single 3D shape as the final output.To tackle this problem, we propose an assembly-oriented dynamic graph learning framework that leverages an iterative graph neural network as a backbone. It explicitly conducts sequential part assembly refinements in a coarse-to-fine manner, exploits a pair of part relation reasoning module and part aggregation module for dynamically adjusting both part features and their relations in the part graph. We conduct extensive experiments and quantitative comparisons to three strong baseline methods, demonstrating the effectiveness of the proposed approach. * Equal Contribution Preprint. Under review.

show abstract

Section: E Additional Results Of Structural Variationmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Generative 3D Part Assembly via Dynamic Graph Learning

Huang,

Zhan,

Fan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using latent 3D model features, the generator expanded the point cloud iteratively in a coarse-to-fine manner, starting from a single point. Wang et al [ 42 ] proposed the part tree to point cloud (PT2PC) model based on conditional generative adversarial networks (GANs). They traversed the input part tree, extracting subtree features bottom-up and recursively decoding part features top-down to generate the final part point cloud.…”

Section: Related Workmentioning

confidence: 99%

Part2Point: A Part-Oriented Point Cloud Reconstruction Framework

Feng,

Zeng,

Liang

2023

Sensors

View full text Add to dashboard Cite

Three-dimensional object modeling is necessary for developing virtual and augmented reality applications. Traditionally, application engineers must manually use art software to edit object shapes or exploit LIDAR to scan physical objects for constructing 3D models. This is very time-consuming and costly work. Fortunately, GPU recently provided a cost-effective solution for massive data computation. With GPU support, many studies have proposed 3D model generators based on different learning architectures, which can automatically convert 2D object pictures into 3D object models with good performance. However, as the demand for model resolution increases, the required computing time and memory space increase as significantly as the parameters of the learning architecture, which seriously degrades the efficiency of 3D model construction and the feasibility of resolution improvement. To resolve this problem, this paper proposes a part-oriented point cloud reconstruction framework called Part2Point. This framework segments the object’s parts, reconstructs the point cloud for individual object parts, and combines the part point clouds into the complete object point cloud. Therefore, it can reduce the number of learning network parameters at the exact resolution, effectively minimizing the calculation time cost and the required memory space. Moreover, it can improve the resolution of the reconstructed point cloud so that the reconstructed model can present more details of object parts.

show abstract

“…Other works focus on leveraging parts structure of clean shapes for co-segmentation (Chen et al, 2019;Zhu et al, 2019), hierarchical mesh segmentation (Yi et al, 2017), shape assembly/generation (Mo et al, 2019a;Wu et al, 2019b;Wu et al, 2019a;Mo et al, 2020), geometry abstraction (Russell et al, 2009;Li et al, 2017;Sun et al, 2019), and other applications.…”

Section: Related Workmentioning

confidence: 99%

Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans

Notchenko

Ishimtsev

Artemov

et al. 2022

Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

We propose Scan2Part, a method to segment individual parts of objects in real-world, noisy indoor RGB-D scans. To this end, we vary the part hierarchies of objects in indoor scenes and explore their effect on scene understanding models. Specifically, we use a sparse U-Net-based architecture that captures the fine-scale detail of the underlying 3D scan geometry by leveraging a multi-scale feature hierarchy. In order to train our method, we introduce the Scan2Part dataset, which is the first large-scale collection providing detailed semantic labels at the part level in the real-world setting. In total, we provide 242,081 correspondences between 53,618 PartNet parts of 2,477 ShapeNet objects and 1,506 ScanNet scenes, at two spatial resolutions of 2 cm 3 and 5 cm 3 . As output, we are able to predict fine-grained per-object part labels, even when the geometry is coarse or partially missing. Overall, we believe that both our method as well as newly introduced dataset is a stepping stone forward towards structural understanding of real-world 3D environments.

show abstract

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

Cited by 4 publications

References 100 publications

Generative 3D Part Assembly via Dynamic Graph Learning

Generative 3D Part Assembly via Dynamic Graph Learning

Part2Point: A Part-Oriented Point Cloud Reconstruction Framework

Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans

Contact Info

Product

Resources

About