Curvature-guided dynamic scale networks for Multi-view Stereo

Abstract

Multi-view stereo (MVS) is a crucial task for precise 3D reconstruction. Most recent studies tried to improve the performance of matching cost volume in MVS by designing aggregated 3D cost volumes and their regularization. This paper focuses on learning a robust feature extraction network to enhance the performance of matching costs without heavy computation in the other steps. In particular, we present a dynamic scale feature extraction network, namely, CDSFNet. It is composed of multiple novel convolution layers, each of which can select a proper patch scale for each pixel guided by the normal curvature of the image surface. As a result, CDFSNet can estimate the optimal patch scales to learn discriminative features for accurate matching computation between reference and source images. By combining the robust extracted features with an appropriate cost formulation strategy, our resulting MVS architecture can estimate depth maps more precisely. Extensive experiments showed that the proposed method outperforms other state-of-the-art methods on complex outdoor scenes. It significantly improves the completeness of reconstructed models. As a result, the method can process higher resolution inputs within faster run-time and lower memory than other MVS methods. Our source code is available at https://github.com/TruongKhang/cds-mvsnet

RobustMVS: Single Domain Generalized Deep Multi-View Stereo

“…Only the training set of DT is selected as source domain to train the model, and we evaluate the performance on other source domains. We directly utilize the open-sourced pretrained model of CasMVSNet [3], PatchMatchNet [59], Iter-MVS [60], MVSTER [47], CDS-MVSNet [61], and UniMVS-Net [5], to test on target domains without finetuning. These pre-trained models are further used for evaulation on unseen datasets on DT → BL, DT → GS, and DT → PA. We further provide experimental results on the same dataset (DT → DT) to evaluate the performance.…”

Feature‐enhanced representation with transformers for multi‐view stereo

“…Simultaneously, high-level features extracted from CNNs exhibit a high level of semantic abstraction, making them well-suited for classification rather than fine-grained feature matching. While some efforts leverage deformable convolutions [9] and normal curvatures [36] to improve the receptive fields in a flexible manner, the extracted features still have inductive biases. Equipped with long-range attention modules, ViTs can provide global perception for MVS models better than the low-level textures, and the patch-wise feature encoding of ViTs also works well for feature matching [37].…”

Adaptive region aggregation for multi‐view stereo matching using deformable convolutional networks

“…Two classical methods (Altizure and OpenMVS) and several deep-learning-based methods from recent years are used for comparative analyses: EPP-MVSNet (Ma et al, 2021), PatchmatchNet (Wang et al, 2020), CDS-MVSNet (Giang et al, 2021) and MG-MVSNet (Zhang et al, 2023). In addition, the results for the original implementation of CasMVSNet (Gu et al, 2019) and TransMVSNet (Ding et al, 2021) are presented.…”