Directional TSDF: Modeling Surface Orientation for Coherent Meshes

Abstract

Real-time 3D reconstruction from RGB-D sensor data plays an important role in many robotic applications, such as object modeling and mapping. The popular method of fusing depth information into a truncated signed distance function (TSDF) and applying the marching cubes algorithm for mesh extraction has severe issues with thin structures: not only does it lead to loss of accuracy, but it can generate completely wrong surfaces. To address this, we propose the directional TSDFa novel representation that stores opposite surfaces separate from each other. The marching cubes algorithm is modified accordingly to retrieve a coherent mesh representation. We further increase the accuracy by using surface gradient-based ray casting for fusing new measurements. We show that our method outperforms state-of-the-art TSDF reconstruction algorithms in mesh accuracy.

Surface Reconstruction Assessment in Photogrammetric Applications

“…The Truncated Signed Distance Field (TSDF) volume integration is a volumetric reconstruction method broadly used while working with low-cost RGB-D sensors and real-time scenarios. It became a standard method since Newcombe et al [ 15 ] used it in the KinectFusion project followed by various extensions and optimizations thereafter [ 79 , 80 , 81 , 82 ]. TSDF methods divide the 3D space (volume) into a discretized set of voxels and fuse distance information into them and is optimized for reconstruction speed.…”

“…On the other hand, in TSDF methods, a truncation threshold is added to omit everything outside this range. The standard method, although efficient under certain scenarios, has some default fundamental limitations as the voxel size itself defines the resolution of the final mesh and anything below this threshold cannot be reconstructed or erroneous results are produced when slanted surfaces are present, requiring alternative optimization solutions (e.g., [ 82 , 84 ]). In this work, M3 uses the TSDF implemented in the Intel Open3D library [ 19 ].…”

Rendering the Directional TSDF for Tracking and Multi-Sensor Registration with Point-To-Plane Scale ICP

“…Especially with thin objects, integration of new measurements might interfere with and contradict old data belonging to a different surface, leading to a corrupted model. We have explored this issue and introduced the concept of the Directional Truncated Signed Distance Function (DTSDF) in our previous work [2]. The DTSDF 1 uses six TSDF volumes, one for each positive and negative coordinate axis, to store surface sections with different orientations.…”

“…In practice, the TSDF is stored as a evenly-spaced grid of voxels and the signed distance, color, and weights in Φ for an arbitrary point in R 3 are estimated by linear interpolation between tuples stored in the neighboring voxels. The directional TSDF [2] Φ dir (p) = (Φ D (p))D∈Directions extends this representation by mapping a point to multiple signed distance functions -one for each direction {X + , X − , Y + , Y − , Z + , Z − } -corresponding to the positive and negative coordinate axes v = {(1, 0, 0) , • • • , (0, 0, −1) }. Observed depth points are assigned to those directions D that fulfill…”

“…While there is no definite weighting scheme, most implementations use a combination of factors to compensate for measurement noise and for uncertainty by down-weighting voxels behind the surface [1,25,14]. In addition to these factors, the fusion weight in [2] includes a direction factor ŵD dir = n, v D to blend surfaces which are represented by multiple directions (c.f. Eq.…”