Weakly Supported Plane Surface Reconstruction via Plane Segmentation Guided Point Cloud Enhancement

Abstract

Most of the widely used multi-view 3D reconstruction algorithms assume that object appearance is predominantly diffuse and full of good texture. For the objects that violate this restriction, the surface can hardly be reconstructed because such area lacks sufficient support from dense point clouds. To tackle this problem, we introduce a novel two-stage prior-guided method based on point clouds enhancement to enable the application of multi-view reconstruction approaches in such scenes. In the first stage, we optimize the original PlaneNet plane segmentation priors by taking advantage of the estimated depth map and confidence map from multi-view stereo. In the second stage, we correct and supply 3D point clouds for the weakly supported plane surface on the basis of the upgraded priors. Furthermore, we utilize a slight disturbance of the enhanced point clouds to facilitate the subsequent mesh reconstruction. The proposed point cloud enhancement approach is evaluated on the large-scale DTU dataset. Our method significantly outperforms previous multi-view stereo state-of-the-arts. We also demonstrate weakly supported plane surface reconstruction results from real-world photos that are unachievable with either the methods aiming at preserving weakly supported surfaces or the traditional state-of-the-art 3D reconstruction systems.

Confidence-Guided Planar-Recovering Multiview Stereo for Weakly Textured Plane of High-Resolution Image Scenes

“…Ref. [19] combines the MVS with PlaneNet to repair incorrect points by correcting and integrating inaccurate prior information from pretrained CNN models and depth map merging methods, then interpolating in weak support planes.…”

“…In structured scenes, surfaces with weakly textured regions can be approximately characterized as identical planes. This allows the plane-based methodology [16][17][18][19][20][21] to effectively guide the elimination of the fuzzy matching problem that occurs in weakly textured regions, then improves the completeness of the reconstruction. Following their previous work, the authors of [18,22] introduce the prior plane to help the recovery of weakly textured regions.…”

Rough Registration of BIM Element Projection for Construction Progress Tracking

“…For example, billions of points need to be processed separately, which consumes computer resources, especially when the background, noise, and objects of no interest need to be deleted [2], [13], [46]. Second, the completeness of the point cloud is difficult to guarantee, and sufficient overlap between images is required to cover all regions of interest [12], [14], [47]. In addition, point clouds have problems such as high noise and difficulty in segmentation and registration [5], [48].…”

Inferring Point Cloud Quality via Graph Similarity

“…These points are often scattered in a 3D space sparsely. Recently, we have witnessed a great deal of efforts that are devoted to drive the point cloud-based applications, such as resampling [3], enhancement [4], [5], saliency detection [6], classification [7], [8], segmentation [9], [10], and compression [2], [11]- [15]. A variety of noises would be inevitably induced by these processing techniques, impairing the reconstruction quality perceived by the human visual system (HVS).…”

“…3 extracts keypoints of a point cloud using the geometry information for local graph construction, color gradient moments aggregation and similarity derivation as discussed in Sec. 4. Experimental studies are conducted in Sec.…”