Analysis of Hand Segmentation in the Wild

Khan, Aisha Urooj; Borji, Ali

doi:10.1109/cvpr.2018.00495

Cited by 40 publications

(23 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 shows the results measured in 4 different object segmentation performance criteria. In comparison to the state-of-the-art hand segmentation networks [21,68] (note [21] is fine-tuned for RHD), our algorithm led to significantly higher accuracy. Also, Fig.…”

Section: Methodsmentioning

confidence: 84%

See 1 more Smart Citation

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering

Baek

Kim

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

209

173

View full text Add to dashboard Cite

Estimating 3D hand meshes from single RGB images is challenging, due to intrinsic 2D-3D mapping ambiguities and limited training data. We adopt a compact parametric 3D hand model that represents deformable and articulated hand meshes. To achieve the model fitting to RGB images, we investigate and contribute in three ways: 1) Neural rendering: inspired by recent work on human body, our hand mesh estimator (HME) is implemented by a neural network and a differentiable renderer, supervised by 2D segmentation masks and 3D skeletons. HME demonstrates good performance for estimating diverse hand shapes and improves pose estimation accuracies. 2) Iterative testing refinement: Our fitting function is differentiable. We iteratively refine the initial estimate using the gradients, in the spirit of iterative model fitting methods like ICP. The idea is supported by the latest research on human body. 3) Self-data augmentation: collecting sized RGB-mesh (or segmentation mask)-skeleton triplets for training is a big hurdle. Once the model is successfully fitted to input RGB images, its meshes i.e. shapes and articulations, are realistic, and we augment view-points on top of estimated dense hand poses. Experiments using three RGB-based benchmarks show that our framework offers beyond state-of-the-art accuracy in 3D pose estimation, as well as recovers dense 3D hand shapes. Each technical component above meaningfully improves the accuracy in the ablation study.

show abstract

Section: Methodsmentioning

confidence: 84%

“…Hand segmentation examples. Left to right: input images, ground-truth masks, our results, the results of the state-of-the-art hand segmentation algorithm[21]. dimension.…”

mentioning

confidence: 99%

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering

Baek

Kim

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

209

173

View full text Add to dashboard Cite

show abstract

“…Multiple studies have explored the use of computer vision techniques to extract information about the hand in egocentric videos, though typically not in the context of rehabilitation. Key problems to be solved include hand detection (locating the hand in the image) as well as segmentation (separating the outline of the hand from the background of the image) [25,24,37,6,15,26,3,41,22,4]. Beyond hand detection and segmentation, there have also been attempts to use egocentric videos for activity recognition and object detection in ADLs [14,16,18,31,33,36].…”

Section: Introductionmentioning

confidence: 99%

Egocentric video: a new tool for capturing hand use of individuals with spinal cord injury at home

Likitlersuang

Sumitro

Cao

et al. 2019

J NeuroEngineering Rehabil

View full text Add to dashboard Cite

Background Current upper extremity outcome measures for persons with cervical spinal cord injury (cSCI) lack the ability to directly collect quantitative information in home and community environments. A wearable first-person (egocentric) camera system is presented that aims to monitor functional hand use outside of clinical settings. Methods The system is based on computer vision algorithms that detect the hand, segment the hand outline, distinguish the user’s left or right hand, and detect functional interactions of the hand with objects during activities of daily living. The algorithm was evaluated using egocentric video recordings from 9 participants with cSCI, obtained in a home simulation laboratory. The system produces a binary hand-object interaction decision for each video frame, based on features reflecting motion cues of the hand, hand shape and colour characteristics of the scene. Results The output from the algorithm was compared with a manual labelling of the video, yielding F1-scores of 0.74 ± 0.15 for the left hand and 0.73 ± 0.15 for the right hand. From the resulting frame-by-frame binary data, functional hand use measures were extracted: the amount of total interaction as a percentage of testing time, the average duration of interactions in seconds, and the number of interactions per hour. Moderate and significant correlations were found when comparing these output measures to the results of the manual labelling, with ρ = 0.40, 0.54 and 0.55 respectively. Conclusions These results demonstrate the potential of a wearable egocentric camera for capturing quantitative measures of hand use at home.

show abstract

“…In , a two-step framework is proposed for detection and segmentation of hands using a Faster R-CNN based hand detector followed by a CNN based skin detection technique. An experimental survey of existing hand segmentation datasets, state-of-the art methods as well as three new datasets for hand segmentation can be found in (Khan and Borji, 2018). Deep learning based methods have been extended to fingertip detection in (Huang et al, 2016).…”

Section: Fingertip Detection and Trackingmentioning

confidence: 99%

Fingertip detection and tracking for recognition of air-writing in videos

Mukherjee

Sekh

Dogra

et al. 2019

Expert Systems with Applications

View full text Add to dashboard Cite

Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1 % while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11 % using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems. IntroductionWith the emergence of virtual and augmented reality, the need for the development of natural human-computer interaction (HCI) systems to replace the traditional HCI approaches is increasing rapidly. In particular, interfaces incorporating hand gesturebased interaction have gained popularity in many fields of application viz. automotive interfaces (Ohn-Bar and Trivedi, 2014), human activity recognition (Rohrbach et al., 2016) and several state-of-the-art hand gesture recognition approaches have been developed (Molchanov et al., 2015;Rautaray and Agrawal, 2015). However, hand motion gestures as such are not sufficient to input text. This necessitates the need for the development of touch-less air-writing systems which may replace touch and electromechanical input panels leading to a more natural human-computer interaction (HCI) approach.A vision-based system for the recognition of mid-air finger-writing trajectories is not a new problem and substantial work has been done in the past two decades. One of the early works by Oka et al. (Oka et al., 2002) used a sophisticated device with an infrared and color sensor for fingertip tracking and recognition of simple geometric shapes trajectories. In (Amma et al., 2012), inertial sensors attached to a glove were used for continuous spotting and recognition of air-writing. Recently, Misra et al. (Misra et al., 2017) have developed a hand gesture recognition framewor...

show abstract

Analysis of Hand Segmentation in the Wild

Cited by 40 publications

References 44 publications

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering

Egocentric video: a new tool for capturing hand use of individuals with spinal cord injury at home

Fingertip detection and tracking for recognition of air-writing in videos

Contact Info

Product

Resources

About