MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

Chen, Anpei; Xu, Zexiang; Zhao, Fuqiang; Zhang, Xiaoshuai; Xiang, Fanbo; Yu, Jingyi; Su, Hao

doi:10.48550/arxiv.2103.15595

Cited by 21 publications

(53 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We train our model on the DTU dataset [18] and evaluate on DTU testing scenes and NeRF synthetic scenes. The results demonstrate that our approach can achieve state-ofthe-art novel view synthesis, outperforming many prior arts including point-based methods [2], NeRF, NSVF [29], and many other generalizable neural methods [8,53,62] (see (Tab. 1 and 2)).…”

Section: Introductionmentioning

confidence: 84%

“…Voxel grids with per-voxel neural features [8,16,29] are also a local neural radiance representation. However, our point-based representation adapts better to actual surfaces, leading to better quality.…”

Section: Related Workmentioning

confidence: 99%

“…This avoids sampling points in the vast empty space and leads to higher rendering quality and faster radiance field reconstruction than PixelNeRF and IBRNet. MVSNeRF [8] can achieve very fast voxel-based radiance field reconstruction. However, its prediction network requires a fixed number of three small-baseline images as input and thus can only efficiently reconstruct local radiance fields.…”

Section: Related Workmentioning

confidence: 99%

“…Since the depth probabilities describe the likelihood of the point being on the surface, we tri-linearly sample the depth probability volume to obtain the point confidence γ i at each point p i . The above process can be expressed by {p i , γ i } = G p,γ (I q , Φ q , I q1 , Φ q1 , I q2 , Φ q2 , ...), (8) where G p,γ is the MVSNet-based network. I q1 , Φ q1 , ... are additional neighboring views used in the MVS reconstruction; we use two additional views in most cases.…”

Section: Generating Initial Point-based Radiance Fieldsmentioning

confidence: 99%

“…Moreover, Point-NeRF avoids ray sampling in the empty scene space by leveraging classical point clouds that approximate the actual scene geometry. This advantage of Point-NeRF leads to more efficient reconstruction and more accurate render-ing than other neural radiance field models [8,35,53,62].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Point-NeRF: Point-based Neural Radiance Fields

Xu¹,

Xu²,

Philip³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Figure 1. Point-NeRF uses neural 3D points to efficiently represent and render a continuous radiance volume. The point-based radiance field can be predicted via network forward inference from multi-view images. It can then be optimized per scene to achieve reconstruction quality that surpasses NeRF [35] in tens of minutes. Point-NeRF can also leverage off-the-shelf reconstruction methods like COLMAP [44]and is able to perform point pruning and growing that automatically fix the holes and outliers that are common in these approaches.

show abstract

Section: Introductionmentioning

confidence: 84%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Generating Initial Point-based Radiance Fieldsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Point-NeRF: Point-based Neural Radiance Fields

Xu¹,

Xu²,

Philip³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Omni-NeRF: Neural Radiance Field from 360° Image Captures

Maugey

Knorr

et al. 2022

2022 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

This paper tackles the problem of novel view synthesis (NVS) from 360 • images with imperfect camera poses or intrinsic parameters. We propose a novel end-to-end framework for training Neural Radiance Field (NeRF) models given only 360 • RGB images and their rough poses, which we refer to as Omni-NeRF. We extend the pinhole camera model of NeRF to a more general camera model that better fits omni-directional fish-eye lenses. The approach jointly learns the scene geometry and optimizes the camera parameters without knowing the fisheye projection.

show abstract

A NeRF-Based Color Consistency Correction Method for Remote Sensing Images

Zuo,

Li,

Zhang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing images are prone to significant variations in photometry due to changing seasons, illumination, and atmospheric conditions. These variations often result in visible stitching seams at the edges of mosaic images, which can impact the visual quality and interpretation of the data. To address color inconsistencies in remote sensing images, conventional methods rely on absolute radiation correction and relative radiation normalization. However, these approaches may not be effective in handling complex variations and producing visually pleasing outcomes. This paper introduces a novel approach based on Neural Radiance Fields (NeRF) for correcting color inconsistencies in multi-view images. Our method leverages implicit expressions and re-illumination of the feature space to capture the intrinsic radiance and reflectance properties of the scene. By intricately weaving image features together, we generate a fusion image that seamlessly integrates color information from multiple views, resulting in improved color consistency and reduced stitching seams. To evaluate the effectiveness of our approach, we conducted experiments using satellite and UAV images with significant variations in range and time. The experimental results demonstrate that our NeRF-based method produces synthesized images with exceptional visual effects and smooth color transitions at the edges. The fusion images exhibit enhanced color consistency, effectively reducing visible stitching seams and elevating the overall image quality.

show abstract

MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

Cited by 21 publications

References 53 publications

Point-NeRF: Point-based Neural Radiance Fields

Point-NeRF: Point-based Neural Radiance Fields

Omni-NeRF: Neural Radiance Field from 360° Image Captures

A NeRF-Based Color Consistency Correction Method for Remote Sensing Images

Contact Info

Product

Resources

About