Instant Neural Graphics Primitives with a Multiresolution Hash Encoding

Abstract

parallelism by implementing the whole system using fully-fused CUDA kernels with a focus on minimizing wasted bandwidth and compute operations. We achieve a combined speedup of several orders of magnitude, enabling training of high-quality neural graphics primitives in a matter of seconds, and rendering in tens of milliseconds at a resolution of 1920×1080.CCS Concepts: • Computing methodologies → Massively parallel algorithms; Vector / streaming algorithms; Neural networks.

Block-NeRF: Scalable Large Scene Neural View Synthesis

“…Several NeRF caching techniques [20,25,72] or a sparse voxel grid [36] could be used to enable real-time Block-NeRF rendering. Similarly, multiple concurrent works have demonstrated techniques to speed up training of NeRF style representations by multiple orders of magnitude [43,60,71].…”

NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing

“…Accelerating NeRF.. There are many existing work to accelerate NeRF [Lindell et al 2020;Liu et al 2020;Müller et al 2022;Reiser et al 2021;Yu et al 2021a,b]. [Liu et al 2020] uses a sparse octree representation with a set of voxel-bounded implicit fields and achieves 10 times faster inference speed compared with the canonical NeRF.…”

“…Recently, ] directly optimizes a sparse 3D grid without any neural networks and achieves more than 100 times faster training speed up and also support real-time rendering. [Müller et al 2022] achieves near-instant training time (around 5s to 1min) of neural graphics primitives with a multi-resolution hash encoding. Though these works are very effective at speeding up NeRF, they only support static scenes.…”

“…For validations, we select a specific frame and conduct static NeRF reconstruction. A number of options are available, from the original NeRF reconstruction [Mildenhall et al 2020], to accelerated Plenoxel , and to the latest, extremely fast NGP [Müller et al 2022]. The reason to test on a static frame is to validate calibration as well as to support conduct better foreground/background segmentation for subsequent frames, to better produce NeuVV.…”

“…By providing real-time rendering capability, NeuVV enables live feedback to the videographer, who can adjust the position, size, and timing of the contents on the fly, greatly improving production efficiency. With the support the latest near real-time neural technique such as NGP [Müller et al 2022], live performance composition and editing in the form of NeuVVs may be practical in foreseeable future. As illustrated in Fig.…”