VIRDO: Visio-tactile Implicit Representations of Deformable Objects

Abstract

Deformable object manipulation requires computationally efficient representations that are compatible with robotic sensing modalities. In this paper, we present VIRDO: an implicit, multi-modal, and continuous representation for deformable-elastic objects. VIRDO operates directly on visual (point cloud) and tactile (reaction forces) modalities and learns rich latent embeddings of contact locations and forces to predict object deformations subject to external contacts. Here, we demonstrate VIRDOs ability to: i) produce highfidelity cross-modal reconstructions with dense unsupervised correspondences, ii) generalize to unseen contact formations, and iii) state-estimation with partial visio-tactile feedback.

iSDF: Real-Time Neural Signed Distance Fields for Robot Perception

“…Other notable works tackle SDF reconstruction in dynamic environments with composite fields [13] and in very large environments using submaps [27]. Lastly, other related directions tackle learning cost fields for motion planning [6,9,33] from SDFs [11,16], navigating in a neural radiance fields [1] and learning neural fields for articulated [18] or deformable objects [35] for manipulation.…”