Shading-based refinement on volumetric signed distance functions

Abstract

Figure 1: Our method obtains fine-scale detail through volumetric shading-based refinement (VSBR) of a distance field. We scan an object using a commodity sensor -here, a PrimeSense -to generate an implicit representation. Unfortunately, this leads to over-smoothing. Exploiting the shading cues from the RGB data allows us to obtain reconstructions at previously unseen resolutions within only a few seconds. AbstractWe present a novel method to obtain fine-scale detail in 3D reconstructions generated with low-budget RGB-D cameras or other commodity scanning devices. As the depth data of these sensors is noisy, truncated signed distance fields are typically used to regularize out the noise, which unfortunately leads to over-smoothed results. In our approach, we leverage RGB data to refine these reconstructions through shading cues, as color input is typically of much higher resolution than the depth data. As a result, we obtain reconstructions with high geometric detail, far beyond the depth resolution of the camera itself. Our core contribution is shading-based refinement directly on the implicit surface representation, which is generated from globally-aligned RGB-D images. We formulate the inverse shading problem on the volumetric distance field, and present a novel objective function which jointly optimizes for fine-scale surface geometry and spatially-varying surface reflectance. In order to enable the efficient reconstruction of sub-millimeter detail, we store and process our surface using a sparse voxel hashing scheme which we augment by introducing a grid hierarchy. A tailored GPU-based Gauss-Newton solver enables us to refine large shape models to previously unseen resolution within only a few seconds.

A Survey of Surface Reconstruction from Point Clouds

“…Follow‐up work has focused on improving the scalability, for instance, through sparse voxel hashing [NZIS13], as well as improving the registration through extraction and matching of stable and robust contour‐based features [ZK15], helping to minimize drift in long capture periods. Recent work [ZDI*15] has demonstrated how to use the high‐resolution images captured in these devices to improve the geometric fidelity of the fused low‐resolution depth images via shading‐based refinement. Using a Lambertian reflectance model, they simultaneously estimate a spatially varying albedo, scene luminance and signed distance directly on the TSDF, which is demonstrated to provide robustness compared to other image or mesh‐based representations (see [ZDI*15], figure ]).…”

“…Recent work [ZDI*15] has demonstrated how to use the high‐resolution images captured in these devices to improve the geometric fidelity of the fused low‐resolution depth images via shading‐based refinement. Using a Lambertian reflectance model, they simultaneously estimate a spatially varying albedo, scene luminance and signed distance directly on the TSDF, which is demonstrated to provide robustness compared to other image or mesh‐based representations (see [ZDI*15], figure ]).…”

Learning Regional Attraction for Line Segment Detection

“…Furthermore, learning signed distance functions has been widely and successfully used for representing 2D closed-boundaries [43], [44], [45] and 3D object surfaces [46], [47]. our proposed attraction field representation shares the similar spirit, but differs in two aspects: Our proposed method directly learns the attraction vectors instead of the distance maps, which can explicitly and accurately characterize the geometry of line segments, and thus eliminates the need of considering the approximation errors for numerical computation.…”

Temporally Consistent Motion Segmentation From RGB‐D Video

“…We enhance the incremental approach with a loop closure strategy to avoid drift similar to Zollhöfer et al . [ZDI*15]. Using temporally more distant frames mirrors the temporal window in our original motion segmentation energy.…”