Model-Free Soft-Structure Reconstruction for Proprioception Using Tactile Arrays

Abstract

Continuum body structures provide unique opportunities for soft robotics, with the infinite degrees of freedom enabling unconstrained and highly adaptive exploration and manipulation. However, the infinite degrees of freedom of continuum bodies makes sensing (both intrinsically and extrinsically) challenging. To address this, in this paper we propose a model-free method for sensorizing tentacle-like continuum soft-structures using an array of spatially arranged capacitive tactile sensors. By using visual tracking, the relationship between the tactile response and the 3D shape of the continuum soft-structure can be learned. A data set of 15000 random soft-body postures was used, with recorded camera-tracked positions logged synchronously to the tactile sensor responses. This was used to train a neural network which can predict posture. We show it is possible to achieve proprioceptive awareness over all three axis of motion in space, reconstructing the body structure and inferring the soft body head's pose with an average accuracy of ≈ 1mm in comparison to the visual tracked counterpart. To demonstrate the capabilities of the system, we perform random exploration of environments limiting the work-space of the sensorized robot. We find the method capable to autonomously reconstruct the reachable morphology of the environment without the need of external sensing units.

A Model-Based Sensor Fusion Approach for Force and Shape Estimation in Soft Robotics

“…The blue matrix plot in the lower part of the figure illustrates the detection of touch points. Overall, these results indicate that measurements in the mutual-capacitive mode are a promising future approach for detecting contact location including multi-touch as well as providing force related readings, similar to traditional tactile sensors [3], [8].…”

“…Regarding shape sensing, some interest is directed towards optical sensing principles, like Ledermann et al, who have studied the use of optical fibers, e. g. [7]. For shape sensing, even capacitive sensing has been studied recently by Scimeca et al [8]. The authors propose to use a capacitive tactile array at the base of a soft finger segment, sensing pressure distribution in order to estimate the pose of the tip using a feed-forward neural network trained using visual tracking data.…”

“…We propose a method which is entirely model-based, i. e. that does not rely on learning, and that leverages this model to fuse multi-modal sensor streams in order to estimate forces and deformations. Learning can achieve high levels of accuracy, e. g. [8]. However, the feature-space used for learning grows exponentially for each additional sensor added to device.…”

Review of machine learning methods in soft robotics

“…Scimeca et al utilized an FNN to learn tactile image information. Moreover, an integration system with a tactile sensor was proposed to obtain internal pressure distributions based on the neural network [ 118 ]. In [ 23 ], a neural network controller for continuum robots was proposed.…”

Gaussian process inference modelling of dynamic robot control for expressive piano playing

“…Advantages of this approach include a mathematically meaningful measure of uncertainty in key-press trajectory prediction. While these implications are valid for both human and robot players, it would be particularly interesting for designers of piano playing robots, because a hard-coded linear mapping of motor control would not be sufficient for human-like playing of piano but an integrative view of morphology and sensory-motor control become more valuable in the context of dexterous manipulation tasks [14][15][16].…”