Crossmodal Source Identification in Speech Perception

Lachs, Lorin; Pisoni, David B.

doi:10.1207/s15326969eco1603_1

Cited by 58 publications

(132 citation statements)

References 53 publications

Supporting

Mentioning

107

Contrasting

Unclassified

Order By: Relevance

“…In fact, initial support for this prediction has been provided by four very recent articles reporting research conducted in parallel with our own study (discussed below; Kamachi, Hill, Lander, & Vatikiotis-Bateson, 2003;Lachs & Pisoni, 2004a, 2004b, 2004c. These four articles reported that, with an XAB methodology, observers were able to successfully match voices to faces and faces to voices.…”

mentioning

confidence: 55%

“…They proposed that in both auditory and visible speech signals, there is speaker-specific articulatory information that specifies both phonetic message and speaker (Kamachi et al, 2003;Lachs & Pisoni, 2004a, 2004b, 2004c. It was thought that this articulatory information conveys an idiosyncratic speaking style that can be specified in the timevarying dimensions of both auditory and visual signals (see also Rosenblum, 2004Rosenblum, , 2005.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Hearing a face: Cross-modal speaker matching using isolated visible speech

Rosenblum

Smith

Nichols

et al. 2006

Perception & Psychophysics

View full text Add to dashboard Cite

An experiment was performed to test whether cross-modal speaker matches could be made using isolated visible speech movement information. Visible speech movements were isolated using a pointlight technique. In five conditions, subjects were asked to match a voice to one of two (unimodal) speaking point-light faces on the basis of speaker identity. Two of these conditions were designed to maintain the idiosyncratic speech dynamics of the speakers, whereas three of the conditions deleted or distorted the dynamics in various ways. Some of these conditions also equated video frames across dynamically correct and distorted movements. The results revealed generally better matching performance in the conditions that maintained the correct speech dynamics than in those conditions that did not, despite containing exactly the same video frames. The results suggest that visible speech movements themselves can support cross-modal speaker matching.

show abstract

mentioning

confidence: 55%

mentioning

confidence: 99%

Hearing a face: Cross-modal speaker matching using isolated visible speech

Rosenblum

Smith

Nichols

et al. 2006

Perception & Psychophysics

View full text Add to dashboard Cite

show abstract

“…However, it is not so trivial to put forward a specific hypothesis about the degree to which visual and auditory information affect the perception of trustworthiness. There exists a large body of research on cross-modal integration in person identification ( [11,12] and many others), but not so much on face-voice information interaction in the case of person evaluation. But since trustworthiness has been shown to be closely related to emotional valence [13], we expected that previous findings on cross-modal integration in the case of emotion expression would apply to the perception of trustworthiness as well.…”

Section: Predictionsmentioning

confidence: 99%

Facial and Vocal Cues in Perceptions of Trustworthiness

Tsankova

Aubrey

Krumhuber

et al. 2013

Computer Vision - ACCV 2012 Workshops

View full text Add to dashboard Cite

Abstract. The goal of the present research was to study the relative role of facial and acoustic cues in the formation of trustworthiness impressions. Furthermore, we investigated the relationship between perceived trustworthiness and perceivers' confidence in their judgments. 25 young adults watched a number of short clips in which the video and audio channel were digitally aligned to form five different combinations of actors' face and voice trustworthiness levels (neutral face + neutral voice, neutral face + trustworthy voice, neutral face + non-trustworthy voice, trustworthy face + neutral voice, and non-trustworthy face + neutral voice). Participants provided subjective ratings of the trustworthiness of the actor in each video, and indicated their level of confidence in each of those ratings. Results revealed a main effect of face-voice channel combination on trustworthiness ratings, and no significant effect of channel combination on confidence ratings. We conclude that there is a clear superiority effect of facial over acoustic cues in the formation of trustworthiness impressions, propose a method for future investigation of the judgment-confidence link, and outline the practical implications of the experiment.

show abstract

“…Visual information about speech influences what listeners hear (Desjardins et al, 1997;Lachs and Pisoni, 2004;McGurk and MacDonald, 1976;MacDonald and McGurk, 1978;MacDonald et al, 2000;Reisberg et al, 1987). This visible articulatory information on a speaker's face is thought to be a central part of typical perceptual development and to foster native language acquisition (Legerstee, 1990) and has been demonstrated in infancy (Burnham and Dodd, 1998;Meltzoff and Kuhl, 1994;Rosenblum et al, 1997).…”

Section: Introductionmentioning

confidence: 99%

The development of gaze to a speaking face

Irwin

Brancazio

Volpe

2017

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

When a speaker talks, the visible consequences of what they are saying can be seen. Listeners are influenced by this visible speech both in a noisy listening environment and even when auditory speech can easily be heard. While visible influence on heard speech has been reported to increase from early to late childhood, little is known about the mechanism that underlies this developmental trend. One possible account of developmental differences is that looking behavior to the face of a speaker changes with age. To examine this possibility, the gaze to a speaking face was examined in children from 5 to 10 yrs of age and adults. Participants viewed a speaker's face in a range of conditions that elicit looking: in a visual only (speech reading) condition, in the presence of auditory noise (speech in noise) condition, and in an audiovisual mismatch (McGurk) condition. Results indicate an increase in gaze on the face, and specifically, to the mouth of a speaker between the ages of 5 and 10 for all conditions. This change in looking behavior may help account for previous findings in the literature showing that visual influence on heard speech increases with development.

show abstract

Crossmodal Source Identification in Speech Perception

Cited by 58 publications

References 53 publications

Hearing a face: Cross-modal speaker matching using isolated visible speech

Hearing a face: Cross-modal speaker matching using isolated visible speech

Facial and Vocal Cues in Perceptions of Trustworthiness

The development of gaze to a speaking face

Contact Info

Product

Resources

About