Understanding Human Hands in Contact at Internet Scale

Shan, Dandan; Geng, Jiaqi; Shu, Michelle; Fouhey, David F.

doi:10.1109/cvpr42600.2020.00989

Cited by 199 publications

(301 citation statements)

References 23 publications

Supporting

Mentioning

297

Contrasting

Order By: Relevance

“…Content may change prior to final publication. without hands, we eliminated frames in which hands appeared by using a hand detector [50] and then used the remained frames for the rendering.…”

Section: Discussionmentioning

confidence: 99%

Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation

et al. 2021

View full text Add to dashboard Cite

Hand segmentation is a crucial task in first-person vision. Since first-person images exhibit strong bias in appearance among different environments, adapting a pre-trained segmentation model to a new domain is required in hand segmentation. Here, we focus on appearance gaps for hand regions and backgrounds separately. We propose (i) foreground-aware image stylization and (ii) consensus pseudolabeling for domain adaptation of hand segmentation. We stylize source images independently for the foreground and background using target images as style. To resolve the domain shift that the stylization has not addressed, we apply careful pseudo-labeling by taking a consensus between the models trained on the source and stylized source images. We validated our method on domain adaptation of hand segmentation from real and simulation images. Our method achieved state-of-the-art performance in both settings. We also demonstrated promising results in challenging multi-target domain adaptation and domain generalization settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Hand-object interaction. Reconstructing hand and object jointly has been studied with both RGB input and RGB-D input [62,71,72,73,78,83,86,87,88,89]. Recently, Hasson et al [27,29] achieved promising results on explicitly modeling the contact by combining a parametric hand model MANO [74], with the mesh based representation for the object.…”

Section: Related Workmentioning

confidence: 99%

Grasping Field: Learning Implicit Representations for Human Grasps

Karunratanakul

Yang

Zhang

et al. 2020

2020 International Conference on 3D Vision (3DV)

168

122

View full text Add to dashboard Cite

show abstract

“…Lee et al [19,20] proposed using hands as a guide to identify an object of interest from a photo taken by people with visual impairment. Shan et al [31] collected a large-scale dataset of hand-object interaction along with annotated bounding boxes of hands and objects in contact with each other. Their proposed system can detect hands and objects in contact with each other from an image.…”

Section: Objects and Hands In First-person Videosmentioning

confidence: 99%

“…We use the state-of-the-art algorithm on hand-held object detection [31] trained on a large-scale image dataset of hand-object interaction collected from first-person video datasets [7,23,33]. Given a video frame, it produces bounding boxes of the hand, contact state (self-contact, other people, portable object, and static object), and its manipulating objects (see Figure 5 (a)).…”

Section: Hand-held Object Detectionmentioning

confidence: 99%

GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery

Yagi

Nishiyasu

Kawasaki

et al. 2021

26th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

People spend an enormous amount of time and effort looking for lost objects. To help remind people of the location of lost objects, various computational systems that provide information on their locations have been developed. However, prior systems for assisting people in finding objects require users to register the target objects in advance. This requirement imposes a cumbersome burden on the users, and the system cannot help remind them of unexpectedly lost objects. We propose GO-Finder ("Generic Object Finder"), a registration-free wearable camera based system for assisting people in finding an arbitrary number of objects based on two key features: automatic discovery of hand-held objects and image-based candidate selection. Given a video taken from a wearable camera, Go-Finder automatically detects and groups hand-held objects to form a visual timeline of the objects. Users can retrieve the last appearance of the object by browsing the timeline through a smartphone app. We conducted a user study to investigate how users benefit from using GO-Finder and confirmed improved accuracy and reduced mental load regarding the object search task by providing clear visual cues on object locations. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools.

show abstract

Understanding Human Hands in Contact at Internet Scale

Cited by 199 publications

References 23 publications

Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation

Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation

Grasping Field: Learning Implicit Representations for Human Grasps

GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery

Contact Info

Product

Resources

About