Face tracking and hand gesture recognition for human-robot interaction

Abstract

Abstract-The interaction between man and machines has become an important topic for the robotic community as it can generalise the use of robots. For active H/R interaction scheme, the robot needs to detect human faces in its vicinity and then interpret canonical gestures of the tracked person assuming this interlocutor has been beforehand identified. In this context, we depict functions suitable to detect and recognise faces in video stream and then focus on face or hand tracking functions.An efficient colour segmentation based on a watershed on the skin-like coloured pixels is proposed. A new measurement model is proposed to take into account both shape and colour cues in the particle filter to track face or hand silhouettes in video stream. An extension of the basic condensation algorithm is proposed to achieve recognition of the current hand posture and automatic switching between multiple templates in the tracking loop.Results of tracking and recognition using are illustrated in the paper and show the process robustness in cluttered environments and in various light conditions. The limits of the method and future works are also discussed.

Detecting Biological Motion for Human–Robot Interaction: A Link between Perception and Action

“…Most 2D video analysis methods for human detection currently adopted in robotics rely on appearance or shape features, for instance detecting faces and hands in the scene (Brethes et al, 2004;Gaschler et al, 2012). However, these approaches have severe limitations as scene complexity grow, for instance when the clutter in the environment increases or the light conditions become more challenging.…”

“…We focus on a method purely based on motion, which does not require any a priori knowledge of human shape or skeleton, nor detecting faces and hands (Brethes et al, 2004;Gaschler et al, 2012).…”

Meeting Modelling in the Context of Multimodal Research

“…There are reliable automatic recognition procedures for hand and face tracking under the right conditions (see e.g. [33]). Therefore the annotation environment can reliably fill in the relative location attributes.…”

Improved data association and occlusion handling for vision-based people tracking by mobile robots

“…Many approaches for people tracking on mobile platform are based on skin colour and face recognition (e.g., [15], [1]). However these methods require persons to be close to and facing the robot so that their hands or faces are visible.…”