MARMVS: Matching Ambiguity Reduced Multiple View Stereo for Efficient Large Scale Scene Reconstruction

Xu, Zhenyu; Liu, Yiguang; Shi, Xuelei; Wang, Ying; Zheng, Yunan

doi:10.1109/cvpr42600.2020.00602

Cited by 40 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, Schönberger et al [26], Xu and Tao [50] both use a forward/backward reprojection error as an additional error term for PatchMatch. MARMVS [54] additionally select the optimal patch scale for each pixel to reduce matching ambiguities. However, these methods focus on speeding up computation and handling textureless regions, but seldom have any strategy for geometric detail preserving, which is exactly the main focus of our method.…”

Section: Depth-map Merging Based Methodsmentioning

confidence: 99%

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

et al. 2021

View full text Add to dashboard Cite

This paper presents an accurate and robust dense 3D reconstruction system for detail preserving surface modeling of large-scale scenes from multi-view images, which we named DP-MVS. Our system performs high-quality large-scale dense reconstruction, which preserves geometric details for thin structures, especially for linear objects. Our framework begins with a sparse reconstruction carried out by an incremental Structure-from-Motion. Based on the reconstructed sparse map, a novel detail preserving PatchMatch approach is applied for depth estimation of each image view. The estimated depth maps of multiple views are then fused to a dense point cloud in a memory-efficient way, followed by a detail-aware surface meshing method to extract the final surface mesh of the captured scene. Experiments on ETH3D benchmark show that the proposed method outperforms other state-of-the-art methods on F1-score, with the running time more than 4 times faster. More experiments on large-scale photo collections demonstrate the effectiveness of the proposed framework for large-scale scene reconstruction in terms of accuracy, completeness, memory saving, and time efficiency.

show abstract

Section: Depth-map Merging Based Methodsmentioning

confidence: 99%

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

et al. 2021

View full text Add to dashboard Cite

show abstract

“…[25] is the first to implement the PatchMatch based algorithm on GPUs, and it uses the black-red pattern in the spatial spreading step. Nowadays, many algorithms [26], [27], [28] have achieved satisfactory performance. The main part of the whole pipeline is as follows: First, initializing the depth and normal values for each pixel using the random technique.…”

Section: Related Workmentioning

confidence: 99%

“…The parameter used in [24] even could not work for some scenes in [39]. The strategy used in [26] first filters source images using τ . τ is the projection point's position displacement when given 3D points a disturb.…”

Section: A Neighboring Image Selectionmentioning

confidence: 99%

See 1 more Smart Citation

PHI-MVS: Plane Hypothesis Inference Multi-view Stereo for Large-Scale Scene Reconstruction

Sun¹,

Zheng²,

Shi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

PatchMatch based Multi-view Stereo (MVS) algorithms have achieved great success in large-scale scene reconstruction tasks. However, reconstruction of texture-less planes often fails as similarity measurement methods may become ineffective on these regions. Thus, a new plane hypothesis inference strategy is proposed to handle the above issue. The procedure consists of two steps: First, multiple plane hypotheses are generated using filtered initial depth maps on regions that are not successfully recovered; Second, depth hypotheses are selected using Markov Random Field (MRF). The strategy can significantly improve the completeness of reconstruction results with only acceptable computing time increasing. Besides, a new acceleration scheme similar to dilated convolution can speed up the depth map estimating process with only a slight influence on the reconstruction. We integrated the above ideas into a new MVS pipeline, Plane Hypothesis Inference Multi-view Stereo (PHI-MVS). The result of PHI-MVS is validated on ETH3D public benchmarks, and it demonstrates competing performance against the state-of-the-art.

show abstract

“…Furthermore, [26,29] use a forwardbackward reprojection error as an additional error term for the PatchMatch estimation. MARMVS [30] additionally estimate the optimal patch scale to reduce matching ambiguities. While these methods generally perform well on a variety of different datasets and can deal with high resolution images, their traditional similarity measures severely limit them in scenarios with reflective surfaces, occlusions and strong lighting changes.…”

Section: Traditional Mvsmentioning

confidence: 99%

BP-MVSNet: Belief-Propagation-Layers for Multi-View-Stereo

Sormann¹,

Knöbelreiter²,

Kühn³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this work, we propose BP-MVSNet, a convolutional neural network (CNN)-based Multi-View-Stereo (MVS) method that uses a differentiable Conditional Random Field (CRF) layer for regularization. To this end, we propose to extend the BP layer [16] and add what is necessary to successfully use it in the MVS setting. We therefore show how we can calculate a normalization based on the expected 3D error, which we can then use to normalize the label jumps in the CRF. This is required to make the BP layer invariant to different scales in the MVS setting. In order to also enable fractional label jumps, we propose a differentiable interpolation step, which we embed into the computation of the pairwise term. These extensions allow us to integrate the BP layer into a multi-scale MVS network, where we continuously improve a rough initial estimate until we get high quality depth maps as a result. We evaluate the proposed BP-MVSNet in an ablation study and conduct extensive experiments on the DTU, Tanks and Temples and ETH3D data sets. The experiments show that we can significantly outperform the baseline and achieve state-of-the-art results.

show abstract

MARMVS: Matching Ambiguity Reduced Multiple View Stereo for Efficient Large Scale Scene Reconstruction

Cited by 40 publications

References 35 publications

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

PHI-MVS: Plane Hypothesis Inference Multi-view Stereo for Large-Scale Scene Reconstruction

BP-MVSNet: Belief-Propagation-Layers for Multi-View-Stereo

Contact Info

Product

Resources

About