3-D Fingertip Touch Force Prediction Using Fingernail Imaging With Automated Calibration

Grieve, Thomas R.; Hollerbach, John M.; Mascaro, Stephen A.

doi:10.1109/tro.2015.2459411

Cited by 16 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an example, normal forces result in a whitening band near the distal end of the finger. It has also been shown that these patterns are common among different humans [29]. Therefore, it is essential to preserve the local spatial information during the feature extraction.…”

Section: B Force Estimation Modelmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Tactile Grasp Force Sensing Using Fingernail Imaging via Deep Neural Networks

Fallahinia¹,

Mascaro²

2021

Preprint

View full text Add to dashboard Cite

This paper has introduced a novel approach for the real-time estimation of 3D tactile forces exerted by human fingertips via vision only. The introduced approach is entirely monocular vision-based and does not require any physical force sensor. Therefore, it is scalable, non-intrusive, and easily fused with other perception systems such as body pose estimation, making it ideal for HRI applications where force sensing is necessary. The introduced approach consists of three main modules: finger tracking for detection and tracking of each individual finger, image alignment for preserving the spatial information in the images, and the force model for estimating the 3D forces from coloration patterns in the images. The model has been implemented experimentally, and the results have shown a maximum RMS error of 8.4% (for the entire range of force levels) along all three directions. The estimation accuracy is comparable to the offline models in the literature, such as EigneNail, while, this model is capable of performing force estimation at 30 frames per second.

show abstract

Section: B Force Estimation Modelmentioning

confidence: 99%

“…The image collection and the force measurement procedures are explained in previous works [33] in detail. As it was noted before, the ground truth aligned images are generated using AAM as described in [29]. The dataset was divided into train, test, and validation sets with the ratios of 80%, 10%, and 10%, respectively.…”

Section: B Datasetmentioning

confidence: 99%

Real-Time Tactile Grasp Force Sensing Using Fingernail Imaging via Deep Neural Networks

Fallahinia¹,

Mascaro²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The fingernail imaging technique, which has been developed in the previous works, is an effective method of 3D fingertip force estimation without imposing any constraint on finger placement [1]. For this method, forces are estimated by analyzing the coloration patterns on fingernail due to the blood flow change in tissues beneath the nail when the fingertip comes in contact with objects.…”

Section: A Fingernail Imagingmentioning

confidence: 99%

“…The training data was collected using an automated calibration platform that was developed in the previous work [1]. The data collection experimental setup includes a Magnetic Device (MLDH) that exerts desired force levels according to a hybrid position-force controller with feedback linearization.…”

Section: A Training Data Collectionmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Constrained and Unconstrained Human Grasp Forces Using Fingernail Imaging and Visual Servoing

Fallahinia¹,

Mascaro²

2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Fingernail imaging has been proven to be effective in prior works [1], [2] for estimating the 3D fingertip forces with a maximum RMS estimation error of 7%. In the current research, fingernail imaging is used to perform unconstrained grasp force measurement on multiple fingers to study human grasping. Moreover, two robotic arms with mounted cameras and a visual tracking system have been devised to keep the human fingers in the camera frame during the experiments. Experimental tests have been conducted for six human subjects under both constrained and unconstrained grasping conditions, and the results indicate a significant difference in force collaboration among the fingers between the two grasping conditions. Another interesting result according to the experiments is that in comparison to constrained grasping, unconstrained grasp forces are more evenly distributed over the fingers and there is less force variation (more steadiness) in each finger force. These results validate the importance of measuring grasp forces in an unconstrained manner in order to study how humans naturally grasp objects.

show abstract

Estimation of Frictional Force Using the Thermal Images of Target Surface During Stroking

Shimomura

Fujiwara

Makino

et al. 2022

Haptics: Science, Technology, Applications

View full text Add to dashboard Cite

We propose a method for estimating the frictional force between a contacted surface and the human touch using thermal video images captured using an infrared thermographic camera. Because this method can estimate force remotely, its application to various situations, in which the measurement is difficult to obtain using conventional contact-based methods, is expected. Furthermore, thermal images have the advantage of measuring physical quantities directly related to frictional force. As a result of machine learning using the measured data from multiple subjects and materials, we succeeded in estimating the frictional force with a high accuracy from the information of the temperature change on the surface. In addition, we account for both the frictional and direct heat transferred between the finger and object affecting the temperature change; therefore, we attempted to improve the accuracy by extracting only frictional heat. Consequently, our method succeeded in improving the accuracy.

show abstract

3-D Fingertip Touch Force Prediction Using Fingernail Imaging With Automated Calibration

Cited by 16 publications

References 23 publications

Real-Time Tactile Grasp Force Sensing Using Fingernail Imaging via Deep Neural Networks

Real-Time Tactile Grasp Force Sensing Using Fingernail Imaging via Deep Neural Networks

Comparison of Constrained and Unconstrained Human Grasp Forces Using Fingernail Imaging and Visual Servoing

Estimation of Frictional Force Using the Thermal Images of Target Surface During Stroking

Contact Info

Product

Resources

About