Active appearance models for photorealistic visual speech synthesis

Relating Objective and Subjective Performance Measures for AAM-Based Visual Speech Synthesis

“…Perhaps the most common method for obtaining a numerical measure of performance using subjective assessment involves asking viewers to score their opinion of a particular aspect of the system on a (typically) five point Likert scale, then reporting the mean opinion score averaged across viewers. Specific aspects that have been tested in this way include the coherency between the audio and visual signals [59], [75], [76], and, more commonly, the naturalness of the talking face [15], [20], [29], [42], [46], [56], [70], [76], [77].…”

“…The aim of this experiment was to compare the perceived naturalness of a feature-driven and unit-driven synthesizer. Naturalness is one of the most widely reported subjective measures [15], [20], [29], [42], [46], [56], [70], [76], [77], and it has the advantage that viewers can take all aspects of the synthesizer into account and form an opinion of the overall sense of the synthesis quality. A limitation is that it does not inform about specific aspects of the system (e.g.…”

“…4) Render: For visual speech synthesis, generally the renderer will take one of two forms: an artist drawn computer graphics (CG) model [31], or an image-based technique [22], [25], [28], [32] that may also include a statistical model of the appearance variation [12], [26], [29], [30], [33]. The choice of renderer will largely be determined by the application.…”

“…Conversely, image-based rendering approaches can achieve close to videorealism, but they lack flexibility: often only the face region is animated, the range of facial gestures may be confined to examples seen during training, and unless complex retargeting methods are employed the identity of the talker is fixed. Hybrid approaches that utilize a statistical appearance model offer a convenient compromise between efficiency and realism [7], [29], [35], [36].…”