Enhanced Soft 3D Reconstruction Method with an Iterative Matching Cost Update Using Object Surface Consensus

Abstract

In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.

Refinement of Inverse Depth Plane in Textureless and Occluded Regions in a Multiview Stereo Matching Scheme

“…The proposed idea is based on the update of the precomputed PSS matching costs in an iterative cost optimization process. In one of our previous researches, we have introduced an MVS method called Enhanced Soft 3D Reconstruction (EnSoft3D) [7]. The EnSoft3D method estimates the object surface consensus, and it is used to update the matching cost in an iterative optimization process.…”

Dual Guided Aggregation Network for Stereo Image Matching

“…According to the workflow in [1], a typical stereo matching process consists of four main steps, namely the matching cost computation, cost aggregation, disparity optimization, and disparity refinement. Each step in this pipeline has been extensively studied, and several advanced methods have been proposed [2][3][4][5][6]. Although this workflow performs well, the stepwise pipeline lacks an overall objective function for global optimization, and thus may suffer errors in each step [7].…”