Novel Tactile-SIFT Descriptor for Object Shape Recognition

Abstract

Using a tactile array sensor to recognize an object often requires multiple touches at different positions. This process is prone to move or rotate the object, which inevitably increases difficulty in object recognition. To cope with the unknown object movement, this paper proposes a new tactile-SIFT descriptor to extract features in view of gradients in the tactile image to represent objects, to allow the features being invariant to object translation and rotation. The tactile-SIFT segments a tactile image into overlapping subpatches, each of which is represented using a dn-dimensional gradient vector, similar to the classic SIFT descriptor. Tactile-SIFT descriptors obtained from multiple touches form a dictionary of k words, and the bagof-words method is then used to identify objects. The proposed method has been validated by classifying 18 real objects with data from an off-the-shelf tactile sensor. The parameters of the tactile-SIFT descriptor, including the dimension size dn and the number of subpatches sp, are studied. It is found that the optimal performance is obtained using an 8-D descriptor with three subpatches, taking both the classification accuracy and time efficiency into consideration. By employing tactile-SIFT, a recognition rate of 91.33% has been achieved with a dictionary size of 50 clusters using only 15 touches.

Iterative Closest Labeled Point for tactile object shape recognition

“…Using a high resolution tactile imager named GelSight, Li et al [16] employed binary descriptors to match features extracted from tactile readings and created tactile maps of objects. In our previous work [17], [18], a new Tactile-SIFT descriptor was proposed based on the Scale Invariant Feature Transform (SIFT) descriptor [19]. In addition to the above hand-drafted features, in [20] unsupervised hierarchical feature learning was applied to extract features from raw tactile data for grasping and object recognition tasks.…”

“…As the sensor interacts with objects, the foam gets compressed and the force is transferred to the sensor; thus the pressure values change. The maximum scanning rate of the sensor is 270 frames/s but a rate of 5 frames/s was used in our experiments because in initial studies the chosen sampling rate was found to be sufficient for the tasks [3], [17]. …”

“…In total, four different features are used and compared, i.e., the Tactile-SIFT descriptors proposed in our previous work [17] and three previous features in the literature, i.e., Zernike moments (the best performing feature used in [15]), normalized Hu's moments [22], raw image moments (up to order 2) [23]. Based on [17], the dictionary size k was set to 50 through the experiments. As shown in Fig.…”

“…6. The recognition rates with tactile information only (BoW) against different number of touches, using our Tactile-SIFT descriptors [17], Zernike moments [15], normalized Hu's moments [22] and raw image moments [23] respectively.…”

Recent progress on tactile object recognition

“…Three kinds of representative objects which are usually used for tactile recognition research. Left: rigid objects used in the study by Luo et al 14 ; middle: textured material used in the study by Sinapov et al 15 ; right: deformable objects used in the study by Drimus et al 16 The figures in left and middle panels have been reproduced with permission from IEEE. The figure in right panel has been reproduced with permission from Elsevier.…”