Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation

Peng, Rui; Wang, Rongjie; Wang, Zhenyu; Lai, Yawen; Wang, Ronggang

doi:10.1109/cvpr52688.2022.00845

Cited by 102 publications

(41 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-Learning-based Furukawa [7] 0.613 0.941 0.777 Tola [38] 0.342 1.190 0.766 Campbell [4] 0.835 0.554 0.695 Gipuma [8] 0.283 0.873 0.578 COLMAP [28] 0.411 0.657 0.534 Ours 0.405 0.381 0.393 Learning-based SurfaceNet [13] 0.450 1.040 0.745 MVSNet [47] 0.396 0.527 0.462 P-MVSNet [21] 0.406 0.434 0.420 R-MVSNet [48] 0.383 0.452 0.417 CasMVSNet [10] 0.325 0.385 0.355 PatchMatchNet [39] 0.427 0.277 0.352 UniMVSNet [26] 0.352 0.278 0.315…”

Section: Methods Acc(mm) Comp(mm) Overallmentioning

confidence: 99%

See 1 more Smart Citation

Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching

Huang¹,

Shi²,

Gong³

2023

Preprint

View full text Add to dashboard Cite

The performance of PatchMatch-based multi-view stereo algorithms depends heavily on the source views selected for computing matching costs. Instead of modeling the visibility of different views, most existing approaches handle occlusions in an ad-hoc manner. To address this issue, we propose a novel visibility-guided pixelwise view selection scheme in this paper. It progressively refines the set of source views to be used for each pixel in the reference view based on visibility information provided by already validated solutions. In addition, the Artificial Multi-Bee Colony (AMBC) algorithm is employed to search for optimal solutions for different pixels in parallel. Inter-colony communication is performed both within the same image and among different images. Fitness rewards are added to validated and propagated solutions, effectively enforcing the smoothness of neighboring pixels and allowing better handling of textureless areas. Experimental results on the DTU dataset show our method achieves state-of-the-art performance among non-learning-based methods and retrieves more details in occluded and low-textured regions.

show abstract

Section: Methods Acc(mm) Comp(mm) Overallmentioning

confidence: 99%

“…Inspired by the success of MVSNet [47], numerous learning-based methods [10,21,26,39,48] had been proposed in recent years and shown outstanding performances. They had been ranked on the top of various MVS datasets [1,17,49].…”

Section: Introductionmentioning

confidence: 99%

Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching

Huang¹,

Shi²,

Gong³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The overview [245] investigates deep learning binocular depth estimation methods and gives a comparison of 16 deep learning depth estimation methods, including the GANet [246], PSMNet [247], and SegStereo [248] in 2018-2019. In recent years, some relatively advanced methods have also appeared, such as PlaneMVS [249], Nerfingmvs [250], and [251,252] etc. The architecture overview of PSMNet is shown in Fig.…”

Section: D Cnnmentioning

confidence: 99%

Brain-Inspired Remote Sensing Interpretation: A Comprehensive Survey

Jiao

Huang

Liu

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Brain-inspired algorithms have become a new trend in next-generation artificial intelligence. Through research on brain science, the intelligence of remote sensing algorithms can be effectively improved. This paper summarizes and analyzes the essential properties of brain cognise learning and the recent advance of remote sensing interpretation. Firstly, this paper introduces the structural composition and the properties of the brain. Then, five represent brain-inspired algorithms are studied, including multiscale geometry analysis, compressed sensing, attention mechanism, reinforcement learning, and transfer learning. Next, this paper summarizes the data types of remote sensing, the development of typical applications of remote sensing interpretation and the implementations of remote sensing, including datasets, software, and hardware. Finally, the top ten open problems and the future direction of brain-inspired remote sensing interpretation are discussed. This work aims to comprehensively review the brain mechanisms and the development of remote sensing and to motivate future research on brain-inspired remote sensing interpretation.

show abstract

“…In recent years, the multi-view stereo 3D reconstruction methods based on deep learning have been widely researched [5] [6], which are mainly aimed at improving the reconstruction accuracy. However, they are still worthy of further improved in the aspect of memory consumption, computing complexity, reconstruction completeness, and reconstruction efficiency [7].…”

Section: Introductionmentioning

confidence: 99%

“…For example, in [13] MVSTR was built by using Transformer architecture to extract the dense features with global context and 3D consistency. In [7] UniMVSNet was constructed by improving the loss functions.Some other MVS improved networks include CVP-MVSNet [14] and MVSNet++ [15]. To improve the 3D reconstruction performance of MVS, some depth networks based on multi-stage deep learning have been proposed, such as MV-GwCNet [9], CasMVSNet [12], UCS-Net [16], PatchmatchNet [4], UniMVSNet [7], and TransMVSNet [17].In [4] the proposed PatchmatchNet network is constructed by introducing the idea of Patchmatch algorithm into the end-to-end deep-learning MVS network.…”

Section: Introductionmentioning

confidence: 99%

Multi-View Three-Dimensional Reconstruction based on a Two-Stage Multi-Level Depth Network for Agriculture Applications

Guo¹,

Shi²,

Jin³

2023

Advanced Information Technologies and Applications

View full text Add to dashboard Cite

To address the problems appearing in multi-view three-dimensional (3D) reconstruction, such as the improvement of the accuracy and completeness of the 3D reconstructed images, a two-stage multi-level depth network is proposed. In the stage 1 of the proposed network, several convolutional block attention modules (CBAMs) are applied in the lateral connections of the feature pyramid network (FPN). This is targeted to enhance the spatial and channel relativity of the different hierarchical feature maps so as to bring more semantic information. In the stage 2, the obtained multi-scale feature maps in the stage 1 are tackled by a set of cascaded processing procedures, such as adaptive propagation, single-trees transform, and matching cost computation. As a result, a depth map could be generated and then be further refined in the processing. Comparing with other state-of-the-art methods, the subjective and objective experiments based on the DTU dataset show that our method performs better result in completeness meanwhile maintaining a considerable overall metric. The investigation of applying the proposed method for reconstructing agricultural crop images was carried out, which is based on a set of self-collected images. The experiment shows that a suitable human visual perception for the images could be obtained.

show abstract

Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation

Cited by 102 publications

References 33 publications

Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching

Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching

Brain-Inspired Remote Sensing Interpretation: A Comprehensive Survey

Multi-View Three-Dimensional Reconstruction based on a Two-Stage Multi-Level Depth Network for Agriculture Applications

Contact Info

Product

Resources

About