Lip synchronization in 3-D model based coding for video-conferencing

Encoding Video Narration as Text

“…SynNar is similar to the Actors system [2] in that it does not rely on any form of 3D modeling or rendering (as in [3] for example) to create the visual appearance of the ''talking head''. Rather, it uses morphing techniques [4][5][6][7] to generate a photo-realistic, smoothly-interpolated video sequence from a set of keyframes which represent typical facial positions for each phoneme in the text.…”

“…Provine and Bruton [3] and Waters and Levergood [8] specifically studied the various positions of major facial features (visemes or visual phonemes) during speech. Provine and Bruton's results suggest that a minimum of 20 different positions of mouth and jaw should be catered for in any facial motion synthesis system, while Waters and Levergood actually used 55 in their commercial system (DECface).…”

“…The restoration is restricted to the area around the mouth, but even there, important features such as jaw and tongue position are not addressed. Moreover, interpolation is restricted to 12 different mouth positions which must cover the 40-50 phonemes [3,17,18] of various English dialects. As a result, the visual representation of ENCODING VIDEO NARRATION AS TEXTmany syllables (in particular, diphthongs) is incorrect to the extent that they would be misinterpreted by a lipreader.…”

Animating song

“…The resulting speech-synchronized animation is of extremely good quality if good motion capture techniques and equipment are combined with a high-quality facial model. The third type of method involves using 2D imageprocessing techniques, 13,28,29 achieving good results for speech synchronization. The character is filmed speaking a corpus that includes all the necessary phonemes or triphones (a combination of three phonemes).…”

Lip synchronization for song

“…Speech recognition techniques [16] can also be used for automated segmentation. The sound track can be a speech waveform [7,22,29,30,31,33] or text [22,28,36]. If required, a waveform is then created from the phonemes.…”