Video Face Tracking and Recognition with Skin Region Extraction and Deformable Template Matching

Abstract

The authors describe a face tracking and recognition system for video indexing that handles variable face poses (left-right and up-down) and deformations due to speech and facial expressions. The system is based on deformable template matching, and employs person-specific templates at near-frontal poses for recognition, and novel person-independent templates at multiple poses on the view-sphere for tracking. Relative to an earlier version that used multiple person-specific templates at multiple (left-right) poses, the new system speeds up processing by (i) restricting attention to skin-color regions; (ii) performing recognition using the person-specific templates at near-frontal poses only; and (iii) tracking at non-frontal poses using the novel person-independent templates. Registration is also simplified since multiple views of each target individual are no longer required, but at the cost of a loss of recognition functionality at poses far from frontal (the system instead “remembers” the identity of each individual from near-frontal matches and tracks between them).

Recognizing persons in television programs using region‐based image features

“…Figure shows a typical approach of the existing methods . It illustrates the process of judging whether or not an input face image corresponds to the same person as a face image stored in a database.…”

“…Next, the face model is adjusted to make it fit the face image. This is done using AdaBoost or template matching . Once the fitting is completed, the points corresponding to each vertex of the 3D model are detected as facial feature points.…”

“…Then, after correcting for distortion (if required) , the local features are calculated from the vicinity of the facial feature points. Typical local features are Gabor features , local binary patterns (LBPs) , scale‐invariant feature transform (SIFT) , and histograms of oriented gradients (HOG) . Finally, the image features calculated from the two face images are compared to decide whether or not they correspond to the same person.…”

“…We also demonstrate its superiority in comparative experiments with six conventional techniques. Figure 2 shows a typical approach of the existing methods [20][21][22]. It illustrates the process of judging whether or not an input face image corresponds to the same person as a face image stored in a database.…”

Real-Time LDCRF-Based Method for Inferring TV Viewer Interest

“…A camera mounted on the TV or in a set-top box produces video showing viewers in front of the TV (and anybody else who is present in its field of view). The video from the mounted camera is first processed by the Face Module, a face tracking and recognition system [14] [15] originally developed for automatic video indexing, i.e. automatically extracting metadata from video.…”