PressureVision: Estimating Hand Pressure from a Single RGB Image

Grady, Patrick; Tang, Chengcheng; Brahmbhatt, Samarth; Twigg, Christopher D.; Wan, Chengde; Hays, James; Kemp, Charles C.

doi:10.1007/978-3-031-20068-7_19

Cited by 17 publications

(7 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We suspect that this is due to significant appearance differences from other surfaces in the training set (Figure 9), including a lack of shadows. As shown in prior work [17], performance is sensitive to shadows. We also observe that ContactLabelNet reports no contact for occluded fingertips.…”

Section: Performance Compared To Baselinesmentioning

confidence: 61%

“…1. Performance compared to PressureVisionNet baseline [17]. Our method outperforms the baselines by a large margin.…”

Section: Discussionmentioning

confidence: 94%

“…In contrast to this prior work, our method uses an external camera to view the whole hand from a distance and deep learning to take advantage of multiple types of cues. As we describe elsewhere, our method builds on PressureVision [17].…”

Section: Related Workmentioning

confidence: 99%

“…• PressureVisionNet: The network from [17], using either the original weights or retrained on our fullylabelled data. This method does not use contact labels.…”

Section: Performance Compared To Baselinesmentioning

confidence: 99%

“…PressureVision [17] showed that computer vision can be used to estimate hand pressure, which could enable new applications, such as touch interfaces for augmented reality. Their deep model, PressureVisionNet, estimates hand pressure from a single RGB image.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Visual Estimation of Fingertip Pressure on Diverse Surfaces using Easily Captured Data

Grady¹,

Collins²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Section: Performance Compared To Baselinesmentioning

confidence: 61%

“…1. Performance compared to PressureVisionNet baseline [17]. Our method outperforms the baselines by a large margin.…”

Section: Discussionmentioning

confidence: 94%

Section: Related Workmentioning

confidence: 99%

“…• PressureVisionNet: The network from [17], using either the original weights or retrained on our fullylabelled data. This method does not use contact labels.…”

Section: Performance Compared To Baselinesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Visual Estimation of Fingertip Pressure on Diverse Surfaces using Easily Captured Data

Grady¹,

Collins²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array

Jiang,

Xu,

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Capturing forceful interaction with deformable objects during manipulation benefits applications like virtual reality, telemedicine, and robotics. Replicating full hand-object states with complete geometry is challenging because of the occluded object deformations. Here, we report a visual-tactile recording and tracking system for manipulation featuring a stretchable tactile glove with 1152 force-sensing channels and a visual-tactile joint learning framework to estimate dynamic hand-object states during manipulation. To overcome the strain interference caused by contact with deformable objects, an active suppression method based on symmetric response detection and adaptive calibration is proposed and achieves 97.6% accuracy in force measurement, contributing to an improvement of 45.3%. The learning framework processes the visual-tactile sequence and reconstructs hand-object states. We experiment on 24 objects from 6 categories including both deformable and rigid ones with an average reconstruction error of 1.8 cm for all sequences, demonstrating a universal ability to replicate human knowledge in manipulating objects with varying degrees of deformability.

show abstract

ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

Fan,

Taheri,

Tzionas

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Figure 1. ARCTIC is a dataset of hands dexterously manipulating articulated objects. The dataset contains videos from both eight 3 rd -person allocentric views (a) and one 1 st -person egocentric view (b), together with accurate ground-truth 3D hand and object meshes, captured with a high-quality motion capture system. ARCTIC goes beyond existing datasets to enable the study of dexterous bimanual manipulation of articulated objects (c) and provides detailed contact information between the hands and objects during manipulation (d-e).

show abstract

PressureVision: Estimating Hand Pressure from a Single RGB Image

Cited by 17 publications

References 61 publications

Visual Estimation of Fingertip Pressure on Diverse Surfaces using Easily Captured Data

Visual Estimation of Fingertip Pressure on Diverse Surfaces using Easily Captured Data

Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array

ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

Contact Info

Product

Resources

About