Optimizing Fingernail Imaging Calibration for 3D Force Magnitude Prediction

Grieve, Thomas R.; Hollerbach, John M.; Mascaro, Stephen A.

doi:10.1109/toh.2015.2468229

Cited by 12 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the camera 9, 13 -we extend this method to a more universal environment. Prior works focused on a certain contact surface material, shape, recording time and object weight; 9,13 in this study, with these variables, we find our approach performs accurately. The results show the first application of video-based finger force prediction.…”

Section: Introductionmentioning

confidence: 53%

“…Prior methods of fingernail image registration are 2D-to-3D registration with a grid pattern and fiducial markers onto the finger, 7 2D-to-3D registration using convolutional neural networks (CNNs), 5 rigid body transformation including the Harris feature point-based method, 3 Canny edge detection, 8 template matching using markers, 4 non-rigid registration fitting a finger model, 3 and active appearance models. 9 Other methods use sensors mounted on the finger 4 or require restrictions such as a bracket to support the hand 7 or the finger. 8 Analysis of "natural" human grasping requires, however, a robust and generally applicable system that does not obstruct movements or otherwise interferes with human performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images

Chen¹,

Westling²,

Edin³

et al. 2019

Robotica

View full text Add to dashboard Cite

The study of dexterous manipulation has provided important insights in humans sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here we describe a method using the deformation and color distribution of the fingernail and its surrounding skin, to estimate the fingertip forces, torques and contact surface curvatures for various objects, including the shape and material of the contact surfaces and the weight of the objects. The proposed method circumvents limitations associated with sensorized objects, gloves or fixed contact surface type. In addition, compared with previous single finger estimation in an experimental environment, we extend the approach to multiple finger force estimation, which can be used for applications such as human grasping analysis. Four algorithms are used, c.q., Gaussian process (GP), Convolutional Neural Networks (CNN), Neural Networks with Fast Dropout (NN-FD) and Recurrent Neural Networks with Fast Dropout (RNN-FD), to model a mapping from images to the corresponding labels. The results further show that the proposed method has high accuracy to predict force, torque and contact surface.

show abstract

Section: Introductionmentioning

confidence: 53%

Section: Introductionmentioning

confidence: 99%

Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images

Chen¹,

Westling²,

Edin³

et al. 2019

Robotica

View full text Add to dashboard Cite

show abstract

“…Active Appearance Models (AAM) [23] has been proposed for 2D image registration along with EigenNail (based on eigenvalue analysis of the fingernail images) [24] for 3D force estimation. However, this approach requires each finger of each human subject to be calibrated individually [25].…”

Section: Related Workmentioning

confidence: 99%

Real-Time Tactile Grasp Force Sensing Using Fingernail Imaging via Deep Neural Networks

Fallahinia¹,

Mascaro²

2021

Preprint

View full text Add to dashboard Cite

This paper has introduced a novel approach for the real-time estimation of 3D tactile forces exerted by human fingertips via vision only. The introduced approach is entirely monocular vision-based and does not require any physical force sensor. Therefore, it is scalable, non-intrusive, and easily fused with other perception systems such as body pose estimation, making it ideal for HRI applications where force sensing is necessary. The introduced approach consists of three main modules: finger tracking for detection and tracking of each individual finger, image alignment for preserving the spatial information in the images, and the force model for estimating the 3D forces from coloration patterns in the images. The model has been implemented experimentally, and the results have shown a maximum RMS error of 8.4% (for the entire range of force levels) along all three directions. The estimation accuracy is comparable to the offline models in the literature, such as EigneNail, while, this model is capable of performing force estimation at 30 frames per second.

show abstract

“…Different approaches can also be taken, e.g. [10] showed that visionbased force sensing could be feasible, while [11] uses an interesting approach that estimates forces on the fingertip from fingernail imaging techniques. However, to the best of our knowledge, there are no tools to retrieve contact points and force information in a wearable manner.…”

Section: Introductionmentioning

confidence: 99%

ExoSense: Measuring Manipulation in a Wearable Manner

Battaglia

Catalano

Grioli

et al. 2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Grasp and manipulation is a complex task, deceivingly simple to accomplish for humans in everyday life, yet challenging to implement in a robotic hand. There is a trend in literature to use information obtained from studies on human grasp for the design and control of robotic manipulators. However, the effectiveness of such approach is dependent on the measurement tools that are available for use with human hands. While there are many sensing solutions that are designed for this purpose, obtaining a complete set of measurements of forces during grasp interaction is still challenging. In this work we aim to bridge this gap by introducing ExoSense, a passive hand exoskeleton. This device can provide position and orientation of the fingertips and, when integrated with the fingertip wearable force/torque sensing system ThimbleSense, a complete characterization of manipulation in terms of generalized forces and position of contacts on each fingertip in a completely wearable and unconstrained manner. After validating the device in terms of end-effector posture measurements and overall accuracy of grasp measurements, we report on a preliminary experiment aiming to show the potentialities of the system to study human internal grasp force variations and for neuroscientific investigation in general.This work was partially supported by the European Community funded projects SOFTPRO and SOMA (contracts 688857 and 645599, respectively), by the ERC Advanced Grant no. 291166 SoftHands.

show abstract

Optimizing Fingernail Imaging Calibration for 3D Force Magnitude Prediction

Cited by 12 publications

References 21 publications

Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images

Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images

Real-Time Tactile Grasp Force Sensing Using Fingernail Imaging via Deep Neural Networks

ExoSense: Measuring Manipulation in a Wearable Manner

Contact Info

Product

Resources

About