Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect

Walker, Stephanie; Bruce, Vicki; O’Malley, Claire

doi:10.3758/bf03208369

Cited by 104 publications

(132 citation statements)

References 28 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…Walker, Bruce, and O'Malley (1995) have reported that the familiarity of faces modulated the degree to which visual information influenced speech perception in the audiovisual "McGurk" illusion (McGurk & MacDonald, 1976). have corroborated an influence of face familiarity on speech reading for static photographs of faces; subjects were found to be more efficient in speech reading for personally familiar faces.…”

mentioning

confidence: 54%

Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces

Schweinberger

Burton

Kelly

1999

Perception & Psychophysics

136

171

View full text Add to dashboard Cite

We investigated whether an asymmetric relationship between the perception of identity and emotional expressions in faces (Schweinberger & Soukup, 1998) may be related to differences in the relative processing speed of identity and expression information. Stimulus faces were morphed across identity within a given emotional expression, or were morphed across emotion within a given identity. In Experiment 1, consistent classifications of these images were demonstrated across a wide range of morphing, with only a relatively narrow category boundary. At the same time, classification reaction times (RTs) reflected the increased perceptual difficulty of the morphed images. In Experiment 2, we investigated the effects of variations in the irrelevant dimension on judgments of faces with respect to a relevant dimension, using a Garner-type speeded classification task. RTs for expression classifications were strongly influenced by irrelevant identity information. In contrast, RTs for identity classifications were unaffected by irrelevant expression information, and this held even for stimuli in which identity was more difficult and slower to discriminate than expression. This suggests that differences in processing speed cannot account for the asymmetric relationship between identity and emotion perception. Theoretical accounts proposing independence of identity and emotion perception are discussed in the light of these findings,

show abstract

mentioning

confidence: 54%

Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces

Schweinberger

Burton

Kelly

1999

Perception & Psychophysics

136

171

View full text Add to dashboard Cite

show abstract

“…Familiarity, and thus identity, can also affect how audiovisual speech is processed. A familiar face matched with an unfamiliar voice is less effective at eliciting the McGurk effect than an unfamiliar face paired with an unfamiliar voice (Walker, Bruce & O'Malley, 1995), suggesting that facial identity and vocal identity are not processed completely independently of one another. A person's facial identity and vocal identity during speech share the same general dynamic temporal patterns.…”

Section: Discussionmentioning

confidence: 93%

The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception

2008

View full text Add to dashboard Cite

During face-to-face conversation the face provides auditory and visual linguistic information, and also conveys information about the identity of the speaker. This study investigated behavioral strategies involved in gathering visual information while watching talking faces. The effects of varying talker identity and varying the intelligibility of speech (by adding acoustic noise) on gaze behavior were measured with an eyetracker. Varying the intelligibility of the speech by adding noise had a noticeable effect on the location and duration of fixations. When noise was present subjects adopted a vantage point that was more centralized on the face by reducing the frequency of the fixations on the eyes and mouth and lengthening the duration of their gaze fixations on the nose and mouth. Varying talker identity resulted in a more modest change in gaze behavior that was modulated by the intelligibility of the speech. Although subjects generally used similar strategies to extract visual information in both talker variability conditions, when noise was absent there were more fixations on the mouth when viewing a different talker every trial as opposed to the same talker every trial. These findings provide a useful baseline for studies examining gaze behavior during audiovisual speech perception and perception of dynamic faces. IntroductionWe see and process faces every day in a wide variety of contexts, from line drawings of faces and static photographs, to dynamic movies and live faces during face-to-face communication. These faces contain a wealth of social, emotional, identity and linguistic information. Although a great deal of information can be gleaned from static faces, the motion of dynamic faces contains information about identity and emotion not present in static faces (Ambadar, Schooler & Cohn, 2005;Hill & Johnson, 2001;Knappmeyer, Thornton & Bülthoff, 2003;O'Toole, Roark, & Abdi, 2002;Lander & Bruce, 2000). Facial motion also contains linguistic information, as evidenced by the fact that silent speechreading is possible (Bernstein, Demorest & Tucker, 2000). Rarely though, is this visual speech information present in the complete absence of auditory speech information and audiovisual speech perception is the natural manner of communication. Visual speech information influences the perception of auditory speech in both perfectly audible conditions (McGurk & MacDonald, 1976, MacDonald & Correspondence: J.N. Buchan, Department of Psychology, Queen's University, Humphrey Hall, 62 Arch Street, Kingston, Ontario, Canada K7L 3N6, Tel: 613-533-6275, E-mail: 2jnb@queensu.ca. Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and al...

show abstract

“…Other research shows that remembered attributes of a visual talker interact with lexical and indexical memory (Sheffert & Fowler, 1995). Familiarity with a talker's face has also been shown to improve the accuracy of visual phoneme identification (Schweinberger & Soukup, 1998), of short-term memory for audiovisual words, and of audiovisual speech integration (Walker, Bruce, & O'Malley, 1995).…”

Section: Interactions Between Indexical and Linguistic Processing Acrmentioning

confidence: 99%

Learning to recognize talkers from natural, sinewave, and reversed speech samples.

Sheffert

Pisoni

Fellowes

et al. 2002

Journal of Experimental Psychology: Human Perception and Perfor

109

100

View full text Add to dashboard Cite

In 5 experiments, the authors investigated how listeners learn to recognize unfamiliar talkers and how experience with specific utterances generalizes to novel instances. Listeners were trained over several days to identify 10 talkers from natural, sinewave, or reversed speech sentences. The sinewave signals preserved phonetic and some suprasegmental properties while eliminating natural vocal quality. In contrast, the reversed speech signals preserved vocal quality while distorting temporally based phonetic properties. The training results indicate that listeners learned to identify talkers even from acoustic signals lacking natural vocal quality. Generalization performance varied across the different signals and depended on the salience of phonetic information. The results suggest similarities in the phonetic attributes underlying talker recognition and phonetic perception.When a talker produces an utterance, the listener simultaneously apprehends the linguistic form of the message as well as the nonlinguistic attributes of the talker's unique vocal anatomy and pronunciation habits. Anatomical and stylistic differences in articulation convey an array of personal or indexical qualities, such as personal identity, sex, approximate age, ethnicity, personality, intentions or emotional state, level of alcohol intoxication, and facial expression (see Bricker

show abstract

Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect

Cited by 104 publications

References 28 publications

Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces

Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces

The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception

Learning to recognize talkers from natural, sinewave, and reversed speech samples.

Contact Info

Product

Resources

About