In-Hand Object Localization Using a Novel High-Resolution Visuotactile Sensor

Cui, Shaowei; Wang, Rui; Hu, Jingyi; Wei, Junhang; Wang, Shuo; Lou, Zheng

doi:10.1109/tie.2021.3090697

Cited by 66 publications

(19 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same group also estimated the in-hand object pose by comparing the geometry pre-annotated based on the object model and the tactile information obtained from pressure sensors embedded on a Barrett robot hand [1]. Li et al [16], and Cui et al [17] used a GelSight sensor to estimate and adjust various workpieces for successful insertion. Pirozzi et al [18] used a photo reflector array to measure the deformation of soft finger pads and estimate the in-hand pose of electric wires.…”

Section: B Tactile-based Methodsmentioning

confidence: 99%

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

et al. 2023

View full text Add to dashboard Cite

This paper proposes a method to estimate the 6D pose of an object grasped by a robot hand using RGB cameras mounted on the palm and visuotactile sensors installed at the fingertips. It can deal with objects made from a wide range of materials thanks to combining the two types of sensors. The method allows a robot to robot to perform in-hand pose estimation while holding the object, eliminating the need for preparatory actions or particular environmental backgrounds. The mechanism at the back of the method includes deep-learning-based background subtraction and denoising auto-encoder-based sensor fusion. With the poses estimated using the proposed method, a robot controller can rectify the grasping uncertainty and adjust the robot motion to move an object toward required goals with satisfying accuracy. We conduct various studies and analyses in the experimental section to understand the proposed method's advantages and disadvantages. The results demonstrate the benefits of the proposed combination and mechanism. They also provide essential knowledge to readers considering using a similar configuration for estimating object poses.

show abstract

Section: B Tactile-based Methodsmentioning

confidence: 99%

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

et al. 2023

View full text Add to dashboard Cite

show abstract

“… Optical flow method [28], [37]  Finite element mode [63], [44]  Neural network [58], [53] mainly using the machine learningbased approaches  Speckle detection [77], [80]  Feature enhancement [81], [82] mainly using the physical model-based approaches  Stereo vision [25], [94]  Virtual stereo vision [23], [26] mainly using the physical model-based approaches  Contact area [24]  2D force distribution [17]  Slip field [79]  Contact area [20]  2D force distribution [86]  Slip field [21]  Contact area [91]  Friction coefficient [23]  2D force distribution [97] 3D tactile perception  Geometric features [46]  3D geometry [18]  3D force distribution [63]  Geometric features [82]  3D geometry [82]  3D force distribution [87]  Geometric features [ commonly used in the field of visuotactile sensing. By using a camera to photograph the marks prepared on the sensor contact elastomer, a tactile image containing the position change of the markers can be obtained, and the tactile information can be further obtained by post-processing and analyzing the tactile image.…”

Section: Common Technologiesmentioning

confidence: 99%

Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2D to 3D: A Review

Li¹,

Li²,

Jiang³

2023

Preprint

View full text Add to dashboard Cite

<p>This article presents a detailed review and categorizing of the marker displacement method (MDM) used in vision-based tactile sensors. Vision-based tactile sensors have been proven to be a promising solution for robot tactile perception. Among such sensors, MDM is one of the most commonly used contact characterization and extraction methods. It uses visual approaches to obtain contact deformation and achieve multimodal tactile perception using physical models and post-processing algorithms. In recent years, many tactile sensors using MDM have been developed. However, the existing research does not strictly distinguish between the different types of methods but is uniformly grouped into MDM. Without differentiation, there might be a lack of systematic and comprehensive guidance in analyzing and optimizing the characteristics of MDM and selecting the most suitable method. This article is the first to classify MDM into three typical categories based on the dimensionality perspective: 2D MDM, 2.5D MDM, and 3D MDM. 2D MDM relies only on the monocular camera to acquire the marker array’s 2D displacement field. 2.5D MDM supplements 2D MDM with selected indirect features reflecting the location of the markers in the third dimension. 3D MDM employs a multi-camera system and can obtain the 3D displacement field using the stereo vision method common. Based on the latest literature, we compare the principles, characteristics, advantages and disadvantages, and applications of the three ways in detail. This work can provide a valuable reference for researchers interested in applying MDM in fields such as vision-based tactile sensors.</p>

show abstract

“…Tactile sensors have been integrated into robotic hands to detect the interaction force and location. 130 However, there is still some limitation to be solved in tactile sensors. This first issue is how to improve the flexibility of tactile sensors.…”

Section: Novel Tactile Sensors For Dexterous Manipulationmentioning

confidence: 99%

A review on sensory perception for dexterous robotic manipulation

Xia¹,

Deng

Fang

et al. 2022

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Sensory perception for dexterous robotic hands is an active research area and recent progress in robotics. Effective dexterous manipulation requires robotic hands to accurately feedback their state or perceive the surrounding environment. This article reviews the state-of-the-art of sensory perception for dexterous robotic manipulation. Two types of sensors, such as intrinsic and extrinsic sensors, are introduced according to their function and layout in robotic hands. These sensors provide rich information to a robotic hand, which contains the posture, the contact information of objects, and the physical information of the environment. Then, a comprehensive analysis of perception methods including planning-level, control-level, and learning-level perceptions is presented. The information obtained from sensory perception can help robotic hands to make decisions effectively. Previously issued reviews mainly focus on the design of tactile senor, while we analyze and discuss the relationship among sensing, perception, and dexterous manipulation. Some potential research topics on sensory perception are also summarized and discussed.

show abstract

In-Hand Object Localization Using a Novel High-Resolution Visuotactile Sensor

Cited by 66 publications

References 33 publications

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors

Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2D to 3D: A Review

A review on sensory perception for dexterous robotic manipulation

Contact Info

Product

Resources

About