Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction

Sun, Cheng; Sun, Minqiang; Chen, Hwann-Tzong

doi:10.1109/cvpr52688.2022.00538

Cited by 578 publications

(216 citation statements)

References 51 publications

Supporting

Mentioning

216

Contrasting

Order By: Relevance

“…To improve the speed, it is feasible to change the MLP, which is the most time consuming part. For example, introducing a voxel grid with a small network for implicit storage and rendering [32], [33], or taking neural 3D point cloud [34], or even abandon the network structure directly [35]. Reducing the number of sampling points can also improve rendering speed [9], [36].…”

Section: B Nerf Series Methodsmentioning

confidence: 99%

DR-WLC: Dimensionality Reduction cognition for object detection and pose estimation by Watching, Learning and Checking

Gao¹,

Xu²,

Jiang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Object detection and pose estimation are difficult tasks in robotics and autonomous driving. Existing object detection and pose estimation methods mostly adopt the samedimensional data for training. For example, 2D object detection usually requires a large amount of 2D annotation data with high cost. Using high-dimensional information to supervise lower-dimensional tasks is a feasible way to reduce datasets size. In this work, the DR-WLC, a dimensionality reduction cognitive model, which can perform both object detection and pose estimation tasks at the same time is proposed. The model only requires 3D model of objects and unlabeled environment images (with or without objects) to finish the training. In addition, a bounding boxes generation strategy is also proposed to build the relationship between 3D model and 2D object detection task. Experiments show that our method can qualify the work without any manual annotations and it is easy to deploy for practical applications. Source code is at https: //github.com/IN2-ViAUn/DR-WLC.

show abstract

Section: B Nerf Series Methodsmentioning

confidence: 99%

DR-WLC: Dimensionality Reduction cognition for object detection and pose estimation by Watching, Learning and Checking

Gao¹,

Xu²,

Jiang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Although the pre-training process is similar to ours, the aimed task, the specific methods used, and the implementation of pretraining are different. In addition to the above extensions, NeRF has been extended for dynamic scenes [37], [38], better rendering effects [39], [40], generalization on multiple scenes [41], [42], [43], [44], [45], faster training or inference speed [46], [47], [48], [49], [50], [51], [52], re-lighting rendering [53], [54], [55], geometry or appearance editing [56], [57], [58], [59], [60] and specifically for processing human bodies [61], [62], [63] and faces [64], [65]. Some works are briefly summarized in [66].…”

Section: Related Workmentioning

confidence: 99%

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

Yuan

Lai

Huang

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

The recently proposed neural radiance fields (NeRF) use a continuous function formulated as a multi-layer perceptron (MLP) to model the appearance and geometry of a 3D scene. This enables realistic synthesis of novel views, even for scenes with view dependent appearance. Many follow-up works have since extended NeRFs in different ways. However, a fundamental restriction of the method remains that it requires a large number of images captured from densely placed viewpoints for high-quality synthesis and the quality of the results quickly degrades when the number of captured views is insufficient. To address this problem, we propose a novel NeRF-based framework capable of high-quality view synthesis using only a sparse set of RGB-D images, which can be easily captured using cameras and LiDAR sensors on current consumer devices. First, a geometric proxy of the scene is reconstructed from the captured RGB-D images. Renderings of the reconstructed scene along with precise camera parameters can then be used to pre-train a network. Finally, the network is fine-tuned with a small number of real captured images. We further introduce a patch discriminator to supervise the network under novel views during fine-tuning, as well as a 3D color prior to improve synthesis quality. We demonstrate that our method can generate arbitrary novel views of a 3D scene from as few as 6 RGB-D images. Extensive experiments show the improvements of our method compared with the existing NeRF-based methods, including approaches that also aim to reduce the number of input images.

show abstract

“…Moreover, IBRNet [44] additionally applied the self-attention operation along the ray to estimate the volumetric density of the points, further slowing the speed. Another group of methods [46,31,36,26] improved the speed of querying and training convergence for a single scene, but these approaches have not been generalized to new scenes. In contrast, our system allows rendering novel views at higher speed while generalizing across various scenes.…”

Section: Related Workmentioning

confidence: 99%

Self-improving Multiplane-to-layer Images for Novel View Synthesis

Solovev¹,

Khakhulin²,

Denis³

2022

Preprint

View full text Add to dashboard Cite

We present a new method for lightweight novel-view synthesis that generalizes to an arbitrary forward-facing scene. Recent approaches are computationally expensive, require per-scene optimization, or produce a memory-expensive representation. We start by representing the scene with a set of fronto-parallel semitransparent planes and afterwards convert them to deformable layers in an end-to-end manner. Additionally, we employ a feed-forward refinement procedure that corrects the estimated representation by aggregating information from input views. Our method does not require any fine-tuning when a new scene is processed and can handle an arbitrary number of views without any restrictions. Experimental results show that our approach surpasses recent models in terms of both common metrics and human evaluation, with the noticeable advantage in inference speed and compactness of the inferred layered geometry.

show abstract

Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction

Cited by 578 publications

References 51 publications

DR-WLC: Dimensionality Reduction cognition for object detection and pose estimation by Watching, Learning and Checking

DR-WLC: Dimensionality Reduction cognition for object detection and pose estimation by Watching, Learning and Checking

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

Self-improving Multiplane-to-layer Images for Novel View Synthesis

Contact Info

Product

Resources

About