Lips, Teeth, and the Benefits of Lipreading

Summerfield, Quentin; Macleod, Alison; McGrath, Matthew; Brooke, M. de L.

doi:10.1016/b978-0-444-87143-5.50019-6

Cited by 59 publications

(33 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have shown the importance of the oral cavity (Smeele, Hahnlen, Stevens, Kuhl, & Meltzoff, 1995) and being able to see the teeth (McGrath, 1985;Summerfield, MacLeod, McGrath, & Brooke, 1989).…”

Section: Limitations Of the Present Studymentioning

confidence: 99%

Similarity structure in visual speech perception and optical phonetic signals

Jiang

Auer

Alwan

et al. 2007

Perception & Psychophysics

View full text Add to dashboard Cite

A complete understanding of visual phonetic perception (lipreading) requires linking perceptual effects to physical stimulus properties. However, the talking face is a highly complex stimulus, affording innumerable possible physical measurements. In the search for isomorphism between stimulus properties and phonetic effects, second-order isomorphism was examined between the perceptual similarities of video-recorded perceptually identified speech syllables and the physical similarities among the stimuli. Four talkers produced the stimulus syllables comprising 23 initial consonants followed by one of three vowels. Six normal-hearing participants identified the syllables in a visual-only condition. Perceptual stimulus dissimilarity was quantified using the Euclidean distances between stimuli in perceptual spaces obtained via multidimensional scaling. Physical stimulus dissimilarity was quantified using face points recorded in three dimensions by an optical motion capture system. The variance accounted for in the relationship between the perceptual and the physical dissimilarities was evaluated using both the raw dissimilarities and the weighted dissimilarities. With weighting and the full set of 3-D optical data, the variance accounted for ranged between 46% and 66% across talkers and between 49% and 64% across vowels. The robust second-order relationship between the sparse 3-D point representation of visible speech and the perceptual effects suggests that the 3-D point representation is a viable basis for controlled studies of first-order relationships between visual phonetic perception and physical stimulus attributes.

show abstract

Section: Limitations Of the Present Studymentioning

confidence: 99%

Similarity structure in visual speech perception and optical phonetic signals

Jiang

Auer

Alwan

et al. 2007

Perception & Psychophysics

View full text Add to dashboard Cite

show abstract

“…Several studies have helped to determine this facial information by leaving visible only an individual facial feature (see, e.g., Benoit, Guiard-Marigny, Le Goff, & Adjoudani, 1996;Berger, Garner, & Sudman, 1971;Cohen, Walker, & Massaro, 1996;Greenberg & Bode, 1968;IJsseldijk, 1992;Larr, 1959;Marassa & Lansing, 1995;McGrath, 1985;Montgomery & Jackson, 1983;Stone, 1957;Summerfield, 1979;Summerfield, MacLeod, McGrath, & Brooke, 1989;Summerfield & McGrath, 1984). For example, Summerfield (1979) presented displays in which the talker's lips were coated with ultraviolet paint so that only the lips could be seen.…”

Section: Introductionmentioning

confidence: 99%

When half a face is as good as a whole: Effects of simple substantial occlusion on visual and audiovisual speech perception

Jordan

Thomas

2011

Atten Percept Psychophys

View full text Add to dashboard Cite

The visible movement of a talker's face is an influential component of speech perception. However, the ability of this influence to function when large areas of the face (~50%) are covered by simple substantial occlusions, and so are not visible to the observer, has yet to be fully determined. In Experiment 1, both visual speech identification and the influence of visual speech on identifying congruent and incongruent auditory speech were investigated using displays of a whole (unoccluded) talking face and of the same face occluded vertically so that the entire left or right hemiface was covered. Both the identification of visual speech and its influence on auditory speech perception were identical across all three face displays. Experiment 2 replicated and extended these results, showing that visual and audiovisual speech perception also functioned well with other simple substantial occlusions (horizontal and diagonal). Indeed, displays in which entire upper facial areas were occluded produced performance levels equal to those obtained with unoccluded displays. Occluding entire lower facial areas elicited some impairments in performance, but visual speech perception and visual speech influences on auditory speech perception were still apparent. Finally, implications of these findings for understanding the processes supporting visual and audiovisual speech perception are discussed.

show abstract

“…Both mouthshape (and the visibility of mouth parts) and mouth movement (the dynamics of mouth actions, including rate of speech) play their part. This insight is confirmed by a range of experimental findings 18,20,22 and by findings in applied telematics which show that speechreading accuracy for audiovisual inputs with auditory dynamic noise falls off as framerate (temporal resolution) of the display of the speaker's face drops from about 30Hz to [8][9][10][11][12]24 In addition, seen rate of speech can be readily discriminated and can directly affect the identification of a heard speech token 5,6 But how does this work? Is one process subsumed in the other?…”

Section: Introductionmentioning

confidence: 70%

Seeing speech in space and time: psychological and neurological findings

Campbell

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

View full text Add to dashboard Cite

Unlike heard speech, some aspects of seen speech can be processed independently of their dynamic (temporal) characteristics. The extent to which speechreading may be supported, separately, by visual mechanisms for the analysis of still and moving events is considered in relation to some experimental findings with normal speaker/viewers, to results with neurological patients with circumscribed lesions to parts of the visual system that support the perception of movement, and to cortical imaging studies.

show abstract

Lips, Teeth, and the Benefits of Lipreading

Cited by 59 publications

References 12 publications

Similarity structure in visual speech perception and optical phonetic signals

Similarity structure in visual speech perception and optical phonetic signals

When half a face is as good as a whole: Effects of simple substantial occlusion on visual and audiovisual speech perception

Seeing speech in space and time: psychological and neurological findings

Contact Info

Product

Resources

About