Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification

Abstract

A critical limitation of current methods based on Neural Radiance Fields (NeRF) is that they are unable to quantify the uncertainty associated with the learned appearance and geometry of the scene. This information is paramount in real applications such as medical diagnosis or autonomous driving where, to reduce potentially catastrophic failures, the confidence on the model outputs must be included into the decision-making process. In this context, we introduce Conditional-Flow NeRF (CF-NeRF), a novel probabilistic framework to incorporate uncertainty quantification into NeRF-based approaches. For this purpose, our method learns a distribution over all possible radiance fields modelling which is used to quantify the uncertainty associated with the modelled scene. In contrast to previous approaches enforcing strong constraints over the radiance field distribution, CF-NeRF learns it in a flexible and fully data-driven manner by coupling Latent Variable Modelling and Conditional Normalizing Flows. This strategy allows to obtain reliable uncertainty estimation while preserving model expressivity. Compared to previous state-of-the-art methods proposed for uncertainty quantification in NeRF, our experiments show that the proposed method achieves significantly lower prediction errors and more reliable uncertainty values for synthetic novel view and depth-map estimation.

Fast Learning Radiance Fields by Shooting Much Fewer Rays

“…In Table IV, we further report numerical comparison with NeRF++ under Light Field (LF) dataset. Note that almost all follow-up approaches [64], [68], [69] reporting results under LF dataset use different dataset splits, thus their results are not comparable with ours. Hence, we only list the results of NeRF++ and ours on this dataset.…”

Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields

“…The variance in pixel space is then used as the uncertainty measure for a particular pixel. Conditional-Flow NeRF (CF-NeRF) [21] by the same authors builds on this work and relaxes some of S-NeRF's constraints on the involved distributions, especially the independence assumption between radiance and density.…”

“…Both methods [20], [21] involve a complex reformulation of the NeRF architecture, rendering process, and its training regime, limiting the applicability of the proposed approach to other NeRF implementations such as Instant NGP [9]. In contrast, the ensembles approach we propose here is extremely simple to implement as it does not require any changes in the underlying NeRF architecture.…”

“…We additionally compare against CF-NeRF [21], new work by the authors of [20]. Since they follow the same evaluation protocol, we can directly compare against the results reported in [21].…”

“…Our paper addresses this gap. Specifically, we show that ensembling [19] is an effective approach to quantify uncertainty in Neural Radiance Fields and that prior work has dismissed the ensembles approach prematurely [20], [21]. Our key insight is that instead of only averaging and calculating variance in the RGB space, a NeRF ensemble should consider an additional epistemic uncertainty [22] term that depends on the densities and termination probabilities along individual rays.…”