Case study of human pointing gestures and their data analysis

Deuerlein, Christian; Müller, Fabian; Heß, Peter

doi:10.1016/j.procir.2020.05.040

Cited by 1 publication

(1 citation statement)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order for the robot to infer the target that the user is pointing to, the robot must accurately estimate the position of the finger and its direction with respect to the robot's coordinate frame. The accuracy is even more important in an environment with poor diversity [24]. For example, when commanded through pointing to go to a specific window out of many in a building, a search and rescue drone must accurately identify the correct one.…”

Section: Introductionmentioning

confidence: 99%

Robust Multi-User In-Hand Object Recognition in Human-Robot Collaboration Using a Wearable Force-Myography Device

Bamani

Kahanowich

Ben-David

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

In communication between humans, gestures are often preferred or complementary to verbal expression since the former offers better spatial referral. Finger pointing gesture conveys vital information regarding some point of interest in the environment. In human-robot interaction, a user can easily direct a robot to a target location, for example, in search and rescue or factory assistance. State-of-the-art approaches for visual pointing estimation often rely on depth cameras, are limited to indoor environments and provide discrete predictions between limited targets. In this paper, we explore the learning of models for robots to understand pointing directives in various indoor and outdoor environments solely based on a single RGB camera. A novel framework is proposed which includes a designated model termed PointingNet. PointingNet recognizes the occurrence of pointing followed by approximating the position and direction of the index finger. The model relies on a novel segmentation model for masking any lifted arm. While state-of-the-art human pose estimation models provide poor pointing angle estimation accuracy of 28 • , PointingNet exhibits mean accuracy of less than 2 • . With the pointing information, the target is computed followed by planning and motion of the robot. The framework is evaluated on two robotic systems yielding accurate target reaching.

show abstract

Section: Introductionmentioning

confidence: 99%