Tactile Perception for Teleoperated Robotic Exploration within Granular Media

Abstract

The sense of touch is essential for locating buried objects when vision-based approaches are limited. We present an approach for tactile perception when sensorized robot fingertips are used to directly interact with granular media particles in teleoperated systems. We evaluate the effects of linear and nonlinear classifier model architectures and three tactile sensor modalities (vibration, internal fluid pressure, fingerpad deformation) on the accuracy of estimates of fingertip contact state. We propose an architecture called the Sparse-Fusion Recurrent Neural Network (SF-RNN) in which sparse features are autonomously extracted prior to fusing multimodal tactile data in a fully connected RNN input layer. The multimodal SF-RNN model achieved 98.7% test accuracy and was robust to modest variations in granular media type and particle size, fingertip orientation, fingertip speed, and object location. Fingerpad deformation was the most informative modality for haptic exploration within granular media while vibration and internal fluid pressure provided additional information with appropriate signal processing. We introduce a real-time visualization of tactile percepts for remote exploration by constructing a belief map that combines probabilistic contact state estimates and fingertip location. The belief map visualizes the probability of an object being buried in the search region and could be used for planning.

Recent Progress in Advanced Tactile Sensing Technologies for Soft Grippers

“…Reproduced with permission. [ 211 ] Copyright 2021, ACM. b) Bioinspired tactile‐olfactory associated intelligent sensory system.…”

“…For instance, Jia et al presented a method for tactile sense when robot fingertips are utilized to directly interact with granular media particles in teleoperated systems. [211] To estimate the contact state of the fingertip, they proposed an architecture called the sparse fusion recurrent neural network (SF-RNN) to extract features from three tactile sensor modalities (vibration, internal fluid pressure, fingerpad deformation) (Figure 9a). The multimodal SF-RNN model achieved 98.7% test accuracy.…”

“…Examples I of perception fusion methods used in soft grippers. a) SF-RNN model architecture for multimodal tactile perception in granular media.Reproduced with permission [211]. Copyright 2021, ACM.…”

Probing with Each Step: How a Walking Crab-like Robot Classifies Buried Cylinders in Sand with Hall-Effect Sensors

“…Robotic classification of objects in granular media through tactile sensing has received little attention. Jia et al demonstrated tactile perception of objects in granular media using a teleoperated robotic finger equipped with a BioTac sensor [ 22 ]. The multimodal sensing (vibration, internal fluid pressure, fingerpad deformation) was shown to provide additional information important for classification compared to single modalities.…”

A Soft Barometric Tactile Sensor to Simultaneously Localize Contact and Estimate Normal Force With Validation to Detect Slip in a Robotic Gripper

“…O VER the past decade, robotic soft tactile sensing technologies emerged, finding their use in many research fields such as object feature extraction and classification [1][2], slip detection [3]- [6], dexterous in-hand manipulation [7] and telepresence [8]. Tactile sensors are particularly valuable in the localization of contacts and estimating contact forces for the purpose of grasping and manipulating fragile, lightweight and deformable objects [9].…”