Deep Vibro-Tactile Perception for Simultaneous Texture Identification, Slip Detection, and Speed Estimation

Abstract

Autonomous dexterous manipulation relies on the ability to recognize an object and detect its slippage. Dynamic tactile signals are important for object recognition and slip detection. An object can be identified based on the acquired signals generated at contact points during tactile interaction. The use of vibrotactile sensors can increase the accuracy of texture recognition and preempt the slippage of a grasped object. In this work, we present a Deep Learning (DL) based method for the simultaneous texture recognition and slip detection. The method detects non-slip and slip events, the velocity, and discriminate textures—all within 17 ms. We evaluate the method for three objects grasped using an industrial gripper with accelerometers installed on its fingertips. A comparative analysis of convolutional neural networks (CNNs), feed-forward neural networks, and long short-term memory networks confirmed that deep CNNs have a higher generalization accuracy. We also evaluated the performance of the highest accuracy method for different signal bandwidths, which showed that a bandwidth of 125 Hz is enough to classify textures with 80% accuracy.

Fabric Classification Using a Finger-Shaped Tactile Sensor via Robotic Sliding

“…While the second type of method collects the tactile signals using sensors sensitive to vibrations. Tactile signals are first transformed into the frequency domain and then both temporal and frequency features are extracted to identify textures as in Fishel and Loeb (2012); Khan et al (2016); Kerr et al (2018); Massalim et al (2020). Instead of classifying the exact type of material, the work proposed by Yuan et al (2018) aims at recognizing 11 different properties from 153 varied pieces of clothes using a convolutional neural network (CNN) based architecture.…”

“…While the second type of method collects the tactile signals using sensors sensitive to vibrations. Tactile signals are first transformed into the frequency domain and then both temporal and frequency features are extracted to identify textures as in Fishel and Loeb ( 2012 ); Khan et al ( 2016 ); Kerr et al ( 2018 ); Massalim et al ( 2020 ).…”

“…A recent work Massalim et al ( 2020 ) tries to not only identify textures but also detect slip and estimate the speed of sliding, using an accelerometer installed on the fingertips of the robotic gripper to record vibration. This work combined multiple deep learning techniques to achieve a decent classification accuracy.…”

Action Conditioned Tactile Prediction: a case study on slip prediction

“…However, these works only focus on grasp success rate by predicting contact and slip events while we focus on manipulation. Tactile based deep neural networks are also used for grasp policy learning [16], slip detection [17], tactile and visual data fusion for grasping [4], and tactile reinforcement learning for grasping [30].…”

Active Manipulation

“…In most of the cases, the tactile sensors used in the first case have a higher bandwidth than the ones used in the second case. The sensors of the first group can detect slippage Massalim et al (2019), recognize textures Fishel and Loeb (2012) or perform both Massalim et al (2020). The sensors of the second group are used in force control, object exploration and manipulation.…”