LAFTER: lips and face real time tracker

Abstract

This paper describes an active-camera real-time system for tracking, shape description, and classification of the human face and mouth using only an SGI Indy computel:The system is based on use of 2 -0 blob features, which are spatially-compact clusters of pixels that are similar in terms of low-level image properties. Patterns of behavior (e.g., facial expressions and head movements) can be classified in real-time using Hidden Markov Model (HMM) methods. The system has been tested on hundreds of users and has demonstrated extremely reliable and accurate pe flormance. Typical class$cation accuracies are near 100%.

A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities

“…Hidden Markov Models (HMMs) [16] and extensions have been one of the most popular modeling techniques. Some of the earliest work was done by Starner and Pentland in [18] where they used HMMs for recognizing hand movements in American Sign Language and by Oliver et al [12] to recognize facial expressions. More complex models, such as Parameterized-HMMs [19], Entropic-HMMs [1], Variable-length HMMs [7], Coupled-HMMs [2], structured HMMs [17] and context-free grammars [8] have been used to recognize more complex activities such as the interaction between two people.…”

Skin color-based video segmentation under time-varying illumination

“…A number of different color spaces have been used; however, normalized RGB [7,12,20,31] and HSV [8,14,22,24] are the most common color spaces used. It has been shown that in addition to being tolerant to minor variations in the illuminant, these color spaces also tend to produce minimum overlap between skin-color and background-color distributions [23].…”

“…Statistical approaches can be subdivided further into: parametric approaches [4,5,7,8,12,22,24,25] and non-parametric approaches [1,14,20,31].…”

“…Some robustness may be achieved via the use of luminance invariant color-spaces [7,12]; however, such an approach can withstand only changes that skin-color distributions undergo within a narrow set of conditions. It has been shown that even in the chromaticity plane skin-color undergoes significant changes when the color temperature of the light source changes [16,21].…”

“…Locating and tracking patches of skin-colored pixels through an image sequence is a tool used in many face recognition and gesture tracking systems [1,4,5,7,8,11,12,13]. Skin-color segmentation, particularly useful for its orientation and size invariance, is usually used for localization in early stages of these higher-level systems.…”