Partially Occluded Hands:

Myanganbayar, Battushig; Mata, Cristina; Dekel, Gil; Katz, Boris; Ben-Yosef, Guy; Barbu, Andrei

doi:10.1007/978-3-030-20873-8_6

Cited by 5 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Body-Hands+Openpose's recall values on each test case of MR 4 is prone to below 50%, and the model is severely affected by vertical and diagonal motion blur, resulting in a dramatic decrease in the number of identified hand poses and their corresponding key points. A slightly better result can be found from MediaPipe hands, except for the results obtained from TC 13 , where the diagonal motion blur still poses the greatest effect on its performance, MediaPipe hands' recall values are all above 50 %. This transformation in the recall values indicates that motion blur managed to confuse the two models to consider a huge number of hands as other objects, which further unveils the domain distribution discrepancies issue and the unstable performance of these models.…”

Section: Resultsmentioning

confidence: 78%

See 1 more Smart Citation

Robustness Evaluation in Hand Pose Estimation Models using Metamorphic Testing

Pu¹,

Chong²,

Lim³

2023

Preprint

View full text Add to dashboard Cite

Hand pose estimation (HPE) is a task that predicts and describes the hand poses from images or video frames. When HPE models estimate hand poses captured in a laboratory or under controlled environments, they normally deliver good performance. However, the real-world environment is complex, and various uncertainties may happen, which could degrade the performance of HPE models. For example, the hands could be occluded, the visibility of hands could be reduced by imperfect exposure rate, and the contour of hands prone to be blurred during fast hand movements. In this work, we adopt metamorphic testing to evaluate the robustness of HPE models and provide suggestions on the choice of HPE models for different applications. The robustness evaluation was conducted on four state-of-the-art models, namely MediaPipe hands, OpenPose, BodyHands, and NSRM hand. We found that on average more than 80% of the hands could not be identified by BodyHands, and at least 50% of hands could not be identified by MediaPipe hands when diagonal motion blur is introduced, while an average of more than 50% of strongly underexposed hands could not be correctly estimated by NSRM hand. Similarly, applying occlusions on only four hand joints will also largely degrade the performance of these models. The experimental results show that occlusions, illumination variations, and motion blur are the main obstacles to the performance of existing HPE models. These findings may pave the way for researchers to improve the performance and robustness of hand pose estimation models and their applications.

show abstract

Section: Resultsmentioning

confidence: 78%

“…To encourage the use of HPE models in practice, it is necessary to fulfil the wide settings of real-world scenarios [3,13,24]. However, the existing literature has found that the performance and robustness of HPE models suffer from occlusion [5,9,25], illumination variations [9], and motion blur [14], which are common in practice.…”

Section: Introductionmentioning

confidence: 99%