DIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing

Liu, Shaohui; Zhang, Yinda; Peng, Songyou; Shi, Boxin; Pollefeys, Marc; Cui, Zhaopeng

doi:10.1109/cvpr42600.2020.00209

Cited by 264 publications

(132 citation statements)

References 25 publications

Supporting

Mentioning

132

Contrasting

Order By: Relevance

“…However, both have fundamental limitations -explicit representations often require fixing the structure's topology and have poor local optima, while discrete volumetric approaches scale poorly to higher resolutions. Instead, several recent approaches implicitly encode a continuous volumetric representation of shape [14,21,39,38,42,43,48,50,54,61] or both shape and view-dependent appearance [41,45,47,59,63,66,69,76,12,73] in the weights of a neural network. These latter approaches overcome the aforementioned limitations and have resulted in impressive novel-view renderings of complex real-world scenes.…”

Section: Related Workmentioning

confidence: 99%

Editing Conditional Radiance Fields

Liu

Zhang

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

153

View full text Add to dashboard Cite

A neural radiance field (NeRF) is a scene model supporting high-quality view synthesis, optimized per scene. In this thesis, we explore enabling user editing of a category-level NeRF -also known as a conditional radiance field -trained on a shape category. Specifically, we introduce a method for propagating coarse 2D user scribbles to the 3D space, to modify the color or shape of a local region. First, we propose a conditional radiance field that incorporates new modular network components, including a shape branch that is shared across object instances. Observing multiple instances of the same category, our model learns underlying part semantics without any supervision, thereby allowing the propagation of coarse 2D user scribbles to the entire 3D region (e.g., chair seat). Next, we propose a hybrid network update strategy that targets specific network components, which balances efficiency and accuracy. During user interaction, we formulate an optimization problem that both satisfies the user's constraints and preserves the original object structure. We demonstrate our approach on various editing tasks over three shape datasets and show that it outperforms prior neural editing approaches. Finally, we edit the appearance and shape of a real photograph and show that the edit propagates to extrapolated novel views.

show abstract

Section: Related Workmentioning

confidence: 99%

Editing Conditional Radiance Fields

Liu

Zhang

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

153

View full text Add to dashboard Cite

show abstract

“…Differentiable Render. Differentiable render [12,[32][33][34] is a novel field which empowers the gradients of 3D objects to be calculated and propagated through images, which attracts increasing attention in academia and industry. The methods of differentiable render can be grouped into four categories, according to the underlying data representation: mesh, voxels, point clouds, and neural implicit functions.…”

Section: D Human Mesh Reconstruction Via Implicit Functionmentioning

confidence: 99%

Relative Pose Estimation for RGB-D Human Input Scans via Implicit Function Reconstruction

Liu

Zhang

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

To achieve a promising performance on relative pose estimation for RGB-D scans, a considerable overlap between two RGB-D inputs is often required for most existing methods. However, in many practical applications for human scans, we often have to estimate the relative poses under arbitrary overlaps, which is challenging for existing methods. To deal with this problem, this paper presents a novel end-to-end and coarse-to-fine optimization method. Our method is self-supervision which firstly combines implicit function reconstruction with differentiable render for RGB-D human input scans at arbitrary overlaps in relative pose estimation. The insight is to take advantage of the underlying human geometry prior as much as possible. First of all, for stable coarse poses, we utilize the implicit function reconstruction to dig out abundant hidden cues from unseen regions in the initialization module. To further refine the poses, the differentiable render is leveraged to establish a self-supervision mechanism in the optimization module, which is independent of standard pipelines for feature extracting and accurate correspondence matching. More importantly, our proposed method is flexible to be extended to multiview input scans. The results and evaluations demonstrate that our optimization module is robust for real-world noisy inputs, and our approach outperforms considerably than standard pipelines in non-overlapping setups.

show abstract

“…Several works represent object shapes by learning an implicit function using neural networks [10,14,21,22,35,37,47,55], which allows for the modeling of arbitrary object topologies with dynamic resolution. Many approaches for learning such implicit functions from various input types were also proposed [32,33,45,48,52,56]. These works focus on rigid objects and do not permit shape deformation.…”

Section: Related Workmentioning

confidence: 99%

A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

Karunratanakul

Spurr

Fan

et al. 2021

2021 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

adrian.spurr,zicong.fan,otmar.hilliges,siyu.tang}@inf.ethz.ch https://korrawe.github.io/halo Figure 1. We introduce a novel neural implicit surface representation of human hands (HALO) that is fully driven by keypoint-based skeleton articulation. Taking 3D keypoints as input, a fully differentiable implicit occupancy representation produces high-fidelity reconstruction of the hand surface (top row). We show that HALO facilitates the conditional generation of articulated hands that grasp 3D objects in a realistic and physically plausible manner (bottom row).

show abstract

DIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing

Cited by 264 publications

References 25 publications

Editing Conditional Radiance Fields

Editing Conditional Radiance Fields

Relative Pose Estimation for RGB-D Human Input Scans via Implicit Function Reconstruction

A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

Contact Info

Product

Resources

About