Using deformable templates to infer visual speech dynamics

Hennecke, Marcus E.; Prasad, Kislaya; Stork, David G.

doi:10.1109/acssc.1994.471518

Cited by 63 publications

(28 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is very simple but the precision is limited (see Figure 1-a). Others authors propose to model the upper lip contour with 2 parabolas [5] or to use quartics [13]. It improves accuracy, but the model is still limited by its rigidity, particularly in the case of asymmetric mouth shape (see Figure 1 The choice of the right model for lip is a great challenge because lip contour is highly deformable.…”

Section: Mouth Modelmentioning

confidence: 99%

Parametric models for facial features segmentation

et al. 2006

View full text Add to dashboard Cite

-In this paper, we are dealing with the problem of facial features segmentation (mouth, eyes and eyebrows). A specific parametric model is defined for each deformable feature, each model being able to take into account all the possible deformations. In order to initialize each model, some characteristic points are extracted on each image to be processed (for example, eyes corners, mouth corners and brows corners). In order to fit the model with the contours to be extracted, a gradient flow (of luminance or chrominance) through the estimated contour is maximized because at each point of the searched contour, the gradient (of luminance or chrominance) is normal. The definition of a model associated to each feature offers the possibility to introduce a regularisation constraint. However, the chosen models are flexible enough to produce realistic contours for the mouth, the eyes and the eyebrows. This facial features segmentation is the first step of a set of multi-media applications.

show abstract

Section: Mouth Modelmentioning

confidence: 99%

Parametric models for facial features segmentation

et al. 2006

View full text Add to dashboard Cite

show abstract

“…Their application to lip-tracking has been described in 29]. The outline of the lips is modeled by a set of hand coded polynomials, which are matched onto the outline of the lips, represented by the image gradient.…”

Section: Model -Based Approachesmentioning

confidence: 99%

Speechreading using Probabilistic Models

Luettin

Thacker

1997

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Abstract. A robust method for locating and tracking lips in gray-level image sequences is described. The method learns patterns of shape variability from a training set which constrains the model during image search to only deform in ways similar to the training examples. Image search is guided by a learned gray-level model which is used to describe the large appearance variability of lips. Such v ariability might be due to di erent individuals, illumination, mouth opening, specularity or visibility of teeth and tongue. Visual speech features are recovered from the tracking results and represent both, shape and intensity information. We describe a speechreading (lip-reading) system, where the extracted features are modeled by Gaussian distributions and their temporal dependencies by Hidden Markov Models. Experimental results are presented for locating lips, tracking lips and speechreading. The database used consists of a broad variety o f s p e a k ers and was recorded in a natural environment with no special lighting or lip markers used. For a speaker independent digit recognition task using visual information only, the system achieved an accuracy about equivalent to that of untrained humans. IDIAP{RR 97-12

show abstract

“…Recent years have seen a dramatic flourishing of the engineering literature on AVSR (Yuhas et al, 1990;Wu et al, 1991;Stork et al, 1992;Bregler et al, 1993b;Cosi et al, 1994;Bregler et al, 1994;Wolff et al, 1994;Hennecke et al, 1994;de Sa, 1994;Movellan, 1995). Current interest on AVSR is in part due to the popularization of digital multimedia tools, its potential application to automatic speech recognition in noisy environments (e.g., car telephony, airplane cockpits, noisy offices), and its links to fundamental theoretical issues in engineering and in cognitive science (Movellan & Chadderdon, 1996).…”

Section: Audio Visual Speech Recognitionmentioning

confidence: 99%

Untitled

Movellan

Mineiro

1998

Machine Learning

View full text Add to dashboard Cite

Abstract. This paper analyzes the issue of catastrophic fusion, a problem that occurs in multimodal recognition systems that integrate the output from several modules while working in non-stationary environments. For concreteness we frame the analysis with regard to the problem of automatic audio visual speech recognition (AVSR), but the issues at hand are very general and arise in multimodal recognition systems which need to work in a wide variety of contexts. Catastrophic fusion is said to have occurred when the performance of a multimodal system is inferior to the performance of some isolated modules, e.g., when the performance of the audio visual speech recognition system is inferior to that of the audio system alone. Catastrophic fusion arises because recognition modules make implicit assumptions and thus operate correctly only within a certain context. Practice shows that when modules are tested in contexts inconsistent with their assumptions, their influence on the fused product tends to increase, with catastrophic results. We propose a principled solution to this problem based upon Bayesian ideas of competitive models and inference robustification. We study the approach analytically on a classic Gaussian discrimination task and then apply it to a realistic problem on audio visual speech recognition (AVSR) with excellent results.

show abstract

Using deformable templates to infer visual speech dynamics

Cited by 63 publications

References 5 publications

Parametric models for facial features segmentation

Parametric models for facial features segmentation

Speechreading using Probabilistic Models

Untitled

Contact Info

Product

Resources

About