Learning Self-prior for Mesh Denoising Using Dual Graph Convolutional Networks

Abstract

This study presents a self-prior-based mesh inpainting framework that requires only an incomplete mesh as input, without the need for any training datasets. Additionally, our method maintains the polygonal mesh format throughout the inpainting process without converting the shape format to an intermediate, such as a voxel grid, a point cloud, or an implicit function, which are typically considered easier for deep neural networks to process. To achieve this goal, we introduce two graph convolutional networks (GCNs): single-resolution GCN (SGCN) and multi-resolution GCN (MGCN), both trained in a self-supervised manner. Our approach refines a watertight mesh obtained from the initial hole filling to generate a completed output mesh. Specifically, we train the GCNs to deform an oversmoothed version of the input mesh into the expected completed shape. To supervise the GCNs for accurate vertex displacements, despite the unknown correct displacements at real holes, we utilize multiple sets of meshes with several connected regions marked as fake holes. The correct displacements are known for vertices in these fake holes, enabling network training with loss functions that assess the accuracy of displacement vectors estimated by the GCNs. We demonstrate that our method outperforms traditional dataset-independent approaches and exhibits greater robustness compared to other deep-learning-based methods for shapes that less frequently appear in shape datasets.

Curvature‐driven Multi‐stream Network for Feature‐preserving Mesh Denoising

“…We have selected the latest representative methods for comparison, traditional methods ($$L_0$$ [HS13], GNF [ZDZ*15]) and data‐driven deep learning methods (CNR [WLT16], DNF [LLZ*20] FGC [AFB20], GeoBi [ZSW*22], DDMP [HYOS22], GCN [SFD*22]), thanks to the open‐source executable program or code provided by these authors. For the sake of fairness, all the experiments are conducted according to the parameters provided by the author or adjusted to the best performance.…”

“…Visual comparison between ours and state‐of‐the‐art algorithms on real scanned meshes (Boy01 and Cone22) of Kinect V2. (a) Noisy, (b) GNF [ZDZ*15], (c) CNR [WLT16], (d) FGC [AFB20], (e) DDMP [HYOS22], (f) GCN [SFD*22], (g) Geobi [ZSW*22], (h) CDMS‐Net (Ours), (i) Ground truth.…”

Learning Self-Prior for Mesh Inpainting Using Self-Supervised Graph Convolutional Networks

“…Smoothness term for facet normals: As Mataev et al [21] reported, the regularization term defined with denoised output can prevent the DIP's neural network from overfitting to noise. In geometry processing, Hattori et al [54] reported a similar finding that facet normal regularization using bilateral normal filtering (BNF) [60] is effective in preserving sharp geometric features such as edges and corners. Following those studies, we introduce a regularizer as the smoothness term for facet normals:…”

“…For techniques for meshes, Hattori et al [54] applied the self-prior to mesh restoration but only focused on mesh denoising. Thus, the self-prior for mesh inpainting has not been sufficiently investigated.…”

“…This is because a GCN can encode local geometric features more effectively when working with vertex displacements. This idea has recently been proposed by Hattori et al [54] for mesh denoising, and they reported that it performed better when applying the DIP framework to mesh denoising. As shown in Fig.…”