No abstract

The vision-based tactile sensor has been proven to be a promising device for sensing tactile information. Among such sensors, the marker displacement method (MDM) is the most common method used in such sensors for representing and extracting contact information. It uses the position field and displacement field of a marker array to characterize the original tactile information, and further achieves multimodal tactile perception through original information processing. This article is the first to classify MDM into three typical categories based on the dimensionality perspective: 2D MDM, 2.5D MDM, and 3D MDM. A comparison study is presented with a focus on the principles, characteristics, applications, and distinctions of these three methods. The latest literature has also been researched as the arguments. Finally, a summary of these three categories is presented as a helpful reference.

<p>This article presents a detailed review and categorizing of the marker displacement method (MDM) used in vision-based tactile sensors. Vision-based tactile sensors have been proven to be a promising solution for robot tactile perception. Among such sensors, MDM is one of the most commonly used contact characterization and extraction methods. It uses visual approaches to obtain contact deformation and achieve multimodal tactile perception using physical models and post-processing algorithms. In recent years, many tactile sensors using MDM have been developed. However, the existing research does not strictly distinguish between the different types of methods but is uniformly grouped into MDM. Without differentiation, there might be a lack of systematic and comprehensive guidance in analyzing and optimizing the characteristics of MDM and selecting the most suitable method. This article is the first to classify MDM into three typical categories based on the dimensionality perspective: 2D MDM, 2.5D MDM, and 3D MDM. 2D MDM relies only on the monocular camera to acquire the marker array’s 2D displacement field. 2.5D MDM supplements 2D MDM with selected indirect features reflecting the location of the markers in the third dimension. 3D MDM employs a multi-camera system and can obtain the 3D displacement field using the stereo vision method common. Based on the latest literature, we compare the principles, characteristics, advantages and disadvantages, and applications of the three ways in detail. This work can provide a valuable reference for researchers interested in applying MDM in fields such as vision-based tactile sensors.</p>

Tactile perception has been a hot topic of research in robotics. Robots sense the shape, material, distributed force, slip during contact, and use the multi-modal contact information to control grasping and manipulation. For vision-based tactile sensors, the contact representation and extraction determine the quality of the raw tactile information, and therefore serve a significant role in the robot perception system. This article highlights for the first time the importance of raw representation and extraction in visuotactile perception, and proposes a new multicolor CMP method for enhancing the performance of vision-based tactile sensors. Based on the principle of continuous marker pattern (CMP), the multicolor CMP method is optimized in the pattern and algorithm design. Regarding information representation, we present a new type of marker pattern based on RGB triangles and a preferred layout. In terms of information extraction, we propose a series of extraction strategies with the adaptive growing algorithm (AGA) and the spin-search algorithm (SSA) as the cores. The experiments reveal that the multicolor CMP method achieves improved precision and reliability compared to the former CMP method.

<p>This article presents a detailed review and categorizing of the marker displacement method (MDM) used in vision-based tactile sensors. Vision-based tactile sensors have been proven to be a promising solution for robot tactile perception. Among such sensors, MDM is one of the most commonly used contact characterization and extraction methods. It uses visual approaches to obtain contact deformation and achieve multimodal tactile perception using physical models and post-processing algorithms. In recent years, many tactile sensors using MDM have been developed. However, the existing research does not strictly distinguish between the different types of methods but is uniformly grouped into MDM. Without differentiation, there might be a lack of systematic and comprehensive guidance in analyzing and optimizing the characteristics of MDM and selecting the most suitable method. This article is the first to classify MDM into three typical categories based on the dimensionality perspective: 2D MDM, 2.5D MDM, and 3D MDM. 2D MDM relies only on the monocular camera to acquire the marker array’s 2D displacement field. 2.5D MDM supplements 2D MDM with selected indirect features reflecting the location of the markers in the third dimension. 3D MDM employs a multi-camera system and can obtain the 3D displacement field using the stereo vision method common. Based on the latest literature, we compare the principles, characteristics, advantages and disadvantages, and applications of the three ways in detail. This work can provide a valuable reference for researchers interested in applying MDM in fields such as vision-based tactile sensors.</p>