Unconstrained Real-time Markerless Hand Tracking for Humanoid Interaction

Abstract

Markerless hand tracking of humans can be applied to a broad range of applications, in robotics, animation and natural human-computer interaction. Traditional motion capture and tracking methods involve the usage of devices such as a data glove, or marker points that are fixed and calibrated on the object to perform tracking. Markerless tracking is free from such needs, and therefore allows for more freedom in movement and spontaneous interaction. In this paper, we analyze how a hand tracking system, which reliably tracks arbitrary hand movements can be implemented. We explored a model based approach that uses particle filters for tracking. In this study we also determine the degree to which the inherent parallel properties of particle filter can be exploited to achieve the goal of real-time tracking. We present the effectiveness of the tracking system via the realtime control of a 20 degrees of freedom dextrous robotic hand.

Model-based hand pose estimation using multiple viewpoint silhouette images and Unscented Kalman Filter

“…Several techniques are available to tackle non-linear filtering including Extended Kalman Filter (EKF), UKF, Gaussian sum filter, grid-based methods, and particle filter. In [16], particle filtering was used to track the hand motion to control a 20 DOF robot hand. A modified EKF through constraint fusion was used in [14] to localize and track an articulated arm while Thayananthan et al [13] used tree-based filtering to determine the hand pose.…”

Markerless tracking using interest window for augmented reality applications

“…Thus, markerless tracking approaches using natural features of the environment to be augmented for tracking is a much more promising approach. It is usually based on the interpretation of a video input stream [8].Visual markerless pose trackers mainly rely on natural feature points (interest points or key points) visible in the user's environment [9]. Usually the computational costs are caused by the tracking process, to align properly the real and virtual objects with respect to each other and to create a realistic illusion of fusion between the two worlds.…”

Mobile robot control using 3D hand pose estimation

“…The 3D-model-based approach [1]- [6] involves extracting local characteristics, or silhouettes, in image recorded using a camera and fitting a 3D hand model constructed beforehand on a computer. While this approach estimates hand shapes highly accurately, it processes self-occlusion poorly and requires long processing time.…”