GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger

Abstract

Vision-based tactile sensors have the potential to provide important contact geometry to localize the objective with visual occlusion. However, it is challenging to measure highresolution 3D contact geometry for a compact robot finger, to simultaneously meet optical and mechanical constraints. In this work, we present the GelSight Wedge sensor, which is optimized to have a compact shape for robot fingers, while achieving highresolution 3D reconstruction. We evaluate the 3D reconstruction under different lighting configurations, and extend the method from 3 lights to 1 or 2 lights. We demonstrate the flexibility of the design by shrinking the sensor to the size of a human finger for fine manipulation tasks. We also show the effectiveness and potential of the reconstructed 3D geometry for pose tracking in the 3D space.

PatchGraph: In-hand tactile tracking with learned surface normals

“…Of these different approaches, the work in [7][8][9][10] is most closely related in terms of the final objective of tracking 3D object poses during in-hand manipulations.These, however, require prior object model information either as offline models [7,9,10] or a database from a simulator [8]. In contrast, we do not require a prior model of the object being tracked.…”

“…Given depth gradients { ∂z ∂x , ∂z ∂y }, we can recover the depth map z(x, y) by integration using a fast Poisson solver with discrete sine transform (DST) [35] as used in prior work [2,7]. For the boundary conditions, we use the mean distance to the undisturbed gel surface.…”

“…For baselines, we picked two different pix2pix architectures, unet and resnet. We also trained a baseline 3-layer MLP 5-32-32-3 with tanh activation on an L2 loss, mapping a single color pixel (r,g,b,x,y) to a surface normal (nx,ny,nz), similar to prior work [7].…”

“…Second, physics priors, such as quasi-static planar pushing models, are less informative in the 3D case. Hence, prior work on in-hand object tracking using tactile feedback has relied on a-priori information about the object being localized, such as 3D global models or a database generated by a simulator [7][8][9][10].…”

ReSkin: versatile, replaceable, lasting tactile skins

“…It also simplifies the replacement process by not requiring the user to disconnect and reconnect individual wires. While optical sensors provide high spatial resolution data, they also require a clear line of sight between the camera and elastomer to observe deformations [1,21,22]. The camera's depth-of-focus puts a hard limit on the minimum distance to the elastomer surface leading to relatively bulky sensor modules.…”