Neural radiosity

Abstract

We introduce Neural Radiosity, an algorithm to solve the rendering equation by minimizing the norm of its residual, similar as in classical radiosity techniques. Traditional basis functions used in radiosity, such as piecewise polynomials or meshless basis functions are typically limited to representing isotropic scattering from diffuse surfaces. Instead, we propose to leverage neural networks to represent the full four-dimensional radiance distribution, directly optimizing network parameters to minimize the norm of the residual. Our approach decouples solving the rendering equation from rendering (perspective) images similar as in traditional radiosity techniques, and allows us to efficiently synthesize arbitrary views of a scene. In addition, we propose a network architecture using geometric learnable features that improves convergence of our solver compared to previous techniques. Our approach leads to an algorithm that is simple to implement, and we demonstrate its effectiveness on a variety of scenes with diffuse and non-diffuse surfaces.

Instant Neural Graphics Primitives with a Multiresolution Hash Encoding

“…If the surface of interest is known a-priori, a data structure such as an octree [Takikawa et al 2021] or sparse grid [Chabra et al 2020;Chibane et al 2020;Hadadan et al 2021;Jiang et al 2020;Liu et al 2020;Peng et al 2020a] can be used to cull away the unused features in the dense grid. However, in the NeRF setting, surfaces only emerge during training.…”

“…Second, natural scenes exhibit smoothness, motivating the use of a multi-resolution decomposition [Chibane et al 2020;Hadadan et al 2021]. Figure 2 (d) shows the result of using an encoding in which interpolated features are stored in eight co-located grids with resolutions from 16 3 to 173 3 .…”

MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction

“…Seminal works [9,40,47] use a single MLP as the scene representation and show impressive object-level reconstruction quality, but they do not scale to more complicated or large-scale scenes due to the limited model capacity. Follow-up works [10,21,39,42,51,65,83] combine an MLP decoder with one or multi-level voxel grids of low-dimensional features. Such hybrid representations are able to better represent fine geometric details and can be evaluated fast.…”

“…Multi-Resolution Feature Grids with MLP Decoder. Instead of using a single feature grid Φ θ , one can also employ multi-resolution feature grids {Φ l θ } L l=1 with resolutions R l [10,21,42,65,83]. The resolutions are sampled in geometric space [42] to combine features at different frequencies:…”

“…Apart from incorporating monocular geometric cues, we provide a systematic study and analysis of state-of-the-art design choices for coordinate-based neural representations in the context of implicit surface reconstruction. More specifically, we investigate the following architectures: a single, large MLP [46,71,75,76], a dense SDF grid [30], a single feature grid [27,36,50,51] and multi-resolution feature grids [10,21,42,65,83]. We observe that MLPs act globally and exhibit an inductive smoothness bias while being computationally expensive to optimize and evaluate.…”