Geometric Granularity Aware Pixel-to-Mesh

Shi, Yue; Ni, Bingbing; Liu, Jinxian; Rong, Dingyi; Ye, Qian; Zhang, Wenjun

doi:10.1109/iccv48922.2021.01285

Cited by 11 publications

(7 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some conditional works [10,51,62,63,65,66] focus on completing full shapes from partial inputs on point clouds with task-specific architectures to encode inputs and decode the global features. Others explore single-view reconstruction [32,47,[54][55][56][57]] and text-driven generation [8,14,30,34] to learn a joint condition-shape distribution with deterministic process. However, most conditional generation methods fail in capturing the multiple output modes [34] and adapting to various tasks.…”

Section: Related Workmentioning

confidence: 99%

3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process

Li¹,

Dou²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

We develop a generalized 3D shape generation prior model, tailored for multiple 3D tasks including unconditional shape generation, point cloud completion, and crossmodality shape generation, etc. On one hand, to precisely capture local fine detailed shape information, a vector quantized variational autoencoder (VQ-VAE) is utilized to index local geometry from a compactly learned codebook based on a broad set of task training data. On the other hand, a discrete diffusion generator is introduced to model the inherent structural dependencies among different tokens. In the meantime, a multi-frequency fusion module (MFM) is developed to suppress high-frequency shape feature fluctuations, guided by multi-frequency contextual information. The above designs jointly equip our proposed 3D shape prior model with high-fidelity, diverse features as well as the capability of cross-modality alignment, and extensive experiments have demonstrated superior performances on various 3D shape generation tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process

Li¹,

Dou²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Its method overview can be found in Figure 2.3 (a). Geometric Granularity Aware Pixel2Mesh [135] is a follow-up of Pixel2Mesh [155] and it can edit the topology of the mesh like [111] by utilizing an error estimator nerwork to identify faces to prone or repair. It applies a keypoint detector to detect keypoints from the ground truth 3D mesh, and then uses the keypoints to regularize the deformed mesh.…”

Section: With 3d Supervisionmentioning

confidence: 99%

A Review of Deep Learning-Powered Mesh Reconstruction Methods

Feng¹

2023

Preprint

View full text Add to dashboard Cite

With the recent advances in hardware and rendering techniques, 3D models have emerged everywhere in our life. Yet creating 3D shapes is arduous and requires significant professional knowledge. Meanwhile, Deep learning has enabled high-quality 3D shape reconstruction from various sources, making it a viable approach to acquiring 3D shapes with minimal effort. Importantly, to be used in common 3D applications, the reconstructed shapes need to be represented as polygonal meshes, which is a challenge for neural networks due to the irregularity of mesh tessellations.In this survey, we provide a comprehensive review of mesh reconstruction methods that are powered by machine learning. We first describe various representations for 3D shapes in the deep learning context. Then we review the development of 3D mesh reconstruction methods from voxels, point clouds, single images, and multi-view images. Finally, we identify several challenges in this field and propose potential future directions.

show abstract

“…With the success of deep learning, a large body of literature began to study the use of deep learning to reconstruct 3D objects from a single colorful image. Existing deep learning-based singleview 3D reconstruction methods can be categorized into four types based on their decoder output representation: implicit function representation [4][5][6][7], voxel representation [8][9][10][11], point cloud representation [12][13][14], and mesh representation [15][16][17][18][19][20][21][22]. In this section, we will conduct a literature review of these different methods.…”

Section: Related Workmentioning

confidence: 99%

“…Existing research on single-view reconstruction employs various 3D shape representations, including implicit functions [4][5][6][7], voxels [8][9][10][11], point clouds [12][13][14], meshes [15][16][17][18][19][20][21][22][23], and others. Voxel-based methods are challenging to reconstruct high-precision shapes due to the large amount of memory and computational time required to apply 3D CNN.…”

Section: Introductionmentioning

confidence: 99%

FAmesh: Generating Frequency Adaptive Meshes from Single Images under 2D Hole Constraints

Wen

2023

Applied Sciences

View full text Add to dashboard Cite

Reconstructing 3D models from a single image has numerous applications in fields such as VR/AR, medical imaging, and gaming. However, most mesh-based methods are limited by the use of 0-genus initial templates, which makes it difficult to reconstruct 3D meshes with complex topologies. Additionally, existing methods often prioritize reconstructing the overall shape and neglect to study local meshes with varying curvatures, resulting in a lack of correct and detailed local features in the generated meshes. This paper proposes a 3D reconstruction framework that transitions from global to local and incorporates MLP and GCN. The framework introduces a mesh pruning strategy under a 2D hole constraint to restore the correct mesh topology. Moreover, the framework fine-tunes local details by separately learning corresponding mapping functions on high-frequency and low-frequency local extended patches. The experiment with the proposed network on the ShapeNet dataset shows that the network has a CD value of 1.763 and an F-score of 85.40. The results from extensive experiments demonstrate that our proposed method outperforms existing methods in topology correction and local detail reconstruction.

show abstract

Geometric Granularity Aware Pixel-to-Mesh

Cited by 11 publications

References 36 publications

3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process

3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process

A Review of Deep Learning-Powered Mesh Reconstruction Methods

FAmesh: Generating Frequency Adaptive Meshes from Single Images under 2D Hole Constraints

Contact Info

Product

Resources

About