Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration

Abstract

This paper introduces "Flocks of Features," a fast tracking method for non-rigid and highly articulated objects such as hands. It combines KLT features and a learned foreground color distribution to facilitate 2D position tracking from a monocular view. The tracker's benefits lie in its speed, its robustness against background noise, and its ability to track objects that undergo arbitrary rotations and vast and rapid deformations. We demonstrate tracker performance on hand tracking with a non-stationary camera in unconstrained indoor and outdoor environments. The tracker yields over threefold improvement over a CamShift tracker in terms of the number of frames tracked before the target was lost, and often more than one order of magnitude improvement in terms of the fractions of particular test sequences tracked successfully.

Template-Based Hand Pose Recognition Using Multiple Cues

“…A popular approach is to work in an intensity-normalized colour space. However, this alone is often not sufficient and a method that is able to initialize and adapt the colour model is required [17,18,5,15]. In this paper the skin colour model is obtained from a frontal face detector, which is run at every kth frame (k = 30) and which does not rely on colour information [19].…”

“…Wu and Huang [13] recognized a number of discrete poses using a learning approach, where a labelled data set is combined with a large unlabeled data set. Kölsch and Turk [5] combined tracking with detection in a real-time interface for a wearable camera system: The global hand position was estimated by tracking a large number of skin-coloured corner features. The detection system was able to detect a particular pose at a fixed orientation using cascaded classifiers.…”

“…The applications in the domain of human computer interaction (HCI) are appealing: Systems have been designed for sign language recognition [1,2,3], navigation and control by hand motion [4,5,6,7], and capturing detailed finger articulation [8,9,10,11,12]. For a given recognition task, it is important to look at the assumptions made in each system in order to determine whether or not the method will work.…”

Real-Time Articulated Hand Detection and Pose Estimation

“…In a purely markerless context, [10] employs a flock of features for tracking, while detection is performed by an AdaBoost classifier [11] trained on Haar features [17]; however, although showing nice robustness properties, both procedures do not provide any articulated pose information, but only the approximate location over the image. The most well-known approaches to articulated tracking in 2D and 3D [14,15,6,2,13,5] are instead based on contours, which provide a rich and precise visual cue, and profit from a large pool of predicted features (contour points and lines) from the previous frame, through dynamical data association and local search.…”

Probabilistic Color and Adaptive Multi-Feature Tracking with Dynamically Switched Priority Between Cues

“…Of the most recent contributions we find: A Kalman filter framework for combining geometric templates, color tracking and blob detection in [13] primarily focussed on tracking vehicles on highways; An adaptive particle filtering technique due to Emilio et al [9] employing color and orientation information to formulate a likelihood as a function of derived cue uncertainties; The "Blackboxes" approach [7] to combining intra and inter state space measurements relying on assumptions like conditional independency of measurements given the state and deterministic mapping between state spaces; The flocks of features and color integration by Mathias et al [6] for hand tracking; And fusion of visual cues with regard to their measurement consistency by Hua et al [4].…”