Total Differential Photometric Mesh Refinement with Self-Adapted Mesh Denoising

Abstract

Variational mesh refinement is a crucial step in multiview 3D reconstruction. Existing algorithms either focus on recovering mesh details or focus on suppressing noise. Approaches with consideration of both are lacking. To address this limitation, we proposed a new variational mesh refinement method named total differential mesh refinement (TDR), which mainly included two improvements. First, the traditional partial-differential photo-consistency gradient used in the variational mesh refinement method was replaced by the proposed total-differential photo-consistency gradient. With consideration of the photo-consistency correlation between adjacent pixels, our method can make photo-consistency achieve a more effective convergence than traditional approaches. Second, we introduced the bilateral normal filter with a novel self-adaptive mesh denoising strategy into the variational mesh refinement. This strategy maintains a balance between detail preservation and effective denoising via the zero-normalized cross-correlation (ZNCC) map. Various experiments demonstrated that our method is superior to traditional variational mesh refinement approaches in both accuracy and denoising effect. Moreover, compared with the mesh generated by open-source and commercial software (Context Capture), our meshes are more detailed, regular, and smooth.

An Efficient and High-Quality Mesh Reconstruction Method with Adaptive Visibility and Dynamic Refinement

“…We propose an improved optimization mesh-reconstruction method, and extensive experiments on BlendedMVS proved that our method can reconstruct a high-quality mesh with higher efficiency, taking only 1/3 of the reconstruction time of OpenMVS [8,29] and 1/10 of the reconstruction time of TDR [5] to complete the reconstruction. 2.…”

“…To validate the effectiveness of our method, we conducted quantitative and qualitative experiments on BlendedMVS [28] and two real-world datasets. We compared our method with two state-of-the-art methods, OpenMVS [8,29] and TDR [5]. The results demonstrated that our method had a higher reconstruction quality than OpenMVS and achieved comparable reconstruction quality with TDR.…”

“…Then, multi-view stereo (MVS) [2,3] generates dense point clouds of the scene. Mesh reconstruction [4,5] transforms these point clouds into 3D mesh models. Finally, texture mapping [6] adds textures to meshes to enhance visual quality.…”

“…In general, mesh reconstruction can be categorized into two types of methods, the spline method [7] and the optimization method [4,5,8]. However, the spline method cannot deal with the noise in the point cloud and usually generates a poor-quality mesh that loses the details of the scene.…”

“…Romanoni and Matteucci [27] considered the self-occlusion problem caused by the depth consistency to select image pairs. Qu et al [5] improved the photo-metric consistency of zero-normalized cross-correlation (ZNCC) through total differentiation and enhanced the mesh quality through adaptive mesh filtering. However, it is still challenging to balance reconstruction quality and reconstruction efficiency in photo-metric refinement.…”