Hearing Faces and Seeing Voices: The Integration and Interaction of Face and Voice Processing

doi:10.5334/pb.ar

Cited by 37 publications

(33 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The distinction between FA and VA may be because people receive 80% information from environments through the visual channel (Zhu, 2000) and distribute more resources on visual senses. Previous researches proposed “Facial Overshadowing effect” (Cook and Wilding, 1997; Stevenage and Neil, 2014) when presenting facial and vocal stimuli together. In particular, the impact of vocal stimuli would become relatively weaker and impaired if they are presented alongside facial stimuli, and facial stimuli would remain dominant.…”

Section: Discussionmentioning

confidence: 99%

“…Compared to VA, FA plays a more vital role in perception of integrated impression (overall attractiveness). Stevenage and Neil (2014) suggested that the auditory channel is a separated but parallel pathway in the process of human perception, despite its relatively weaker functioning compared to the visual channel. Rezlescu et al (2015) also supported that people evaluated faces as more attractive than voices in human mating.…”

Section: Introductionmentioning

confidence: 99%

“…According to Smith et al (2016), people generally have more than 50% chance of making correct judgments on whether a multimodal stimulus (faces and voices) belongs to the same person. Moreover, multimodal stimuli combined with same-individual faces and voices are more attractive than those combined with different-individual faces and voices (Stevenage and Neil, 2014). Based on these findings, we predicted that “original multimodal stimuli (OMS)” (multimodal stimuli consist of same-individual faces and voices) would be evaluated as more attractive than “non-original multimodal stimuli (non-OMS)” (multimodal stimuli consist of different-individual faces and voices)—we call this phenomenon as “OMS-facilitating effect.”…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sex Differences in Facial and Vocal Attractiveness Among College Students in China

Hou

2019

Front. Psychol.

View full text Add to dashboard Cite

This study aims to investigate sex differences in ratings for facial attractiveness (FA) and vocal attractiveness (VA). Participants (60 undergraduates in Study 1 and 111 undergraduates in Study 2) rated the attractiveness of computerized face images and voice recordings of men and women. In Study 1, face images and voice recordings were presented separately. Results indicated that men generally rated voice recordings of women more attractive than those of men, whereas women did not show different attractiveness ratings for voices of men vs. women. In Study 2, face images and voice recordings were paired as multimodal stimuli and presented simultaneously. Results indicated that men rated multimodal stimuli of women as more attractive than those of men, whereas women did not differentiate multimodal stimuli of men vs. women. We found that, compared to VA, FA had a stronger influence on participants' overall evaluations. Finally, we tested the difference between “original multimodal stimuli” (OMS) and “non-original multimodal stimuli” (non-OMS) and found the “OMS-facilitating effect.” Taken together, findings indicated some sex differences in FA and VA in the current study, which could be used to interpret behaviors of sexual selection, human mate preferences, and designs and popularization of sex robots.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sex Differences in Facial and Vocal Attractiveness Among College Students in China

Hou

2019

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Possible order effects warrant further investigation, particularly when including intervals of an unprecedented duration (>1 s). The rationale for manipulating the order of stimulus presentation expressed in other studies (see Lachs & Pisoni, 2004a) focuses on face–voice asymmetries in terms of speech information, but it is also possible that differential memory for faces and voices will affect performance when the ISI is longer than 1 s. Voices are less well remembered (Stevenage, Hugill, & Lewis, 2012; Stevenage & Neil, 2014), and more sensitive to interference (Stevenage, Howland, & Tippelt, 2011) than faces. Therefore, it might be the case that performance is less accurate in the A–V condition when it is necessary to remember the voice for longer than the face.…”

Section: Methodsmentioning

confidence: 99%

The effect of inserting an inter-stimulus interval in face–voice matching tasks

Smith

Dunn

Baguley

et al. 2018

Quarterly Journal of Experimental Psychology

View full text Add to dashboard Cite

Based on the results of three experiments, Smith, Dunn, Baguley, and Stacey (2016b) concluded that source identity information is shared by voices and faces regardless of whether the faces are static or dynamic (i.e., articulating but muted). The balance of evidence suggests that voices and static faces do provide sufficient concordant identity information (Smith et al., 2016a) so that it is possible to accurately match an unfamiliar face to a voice (Smith et al., 2016b). All previous tests of face-voice matching have presented faces and voices close together in time, with a maximum 1-s inter-stimulus interval (ISI;

show abstract

“…The human brain achieves this by making use of cues from several sensory modalities, including visual signals from the face of a person and auditory signals from her voice12. There is evidence that these cues are combined across senses to yield more accurate, more robust representations of person identity—a clear case of multisensory integration345. For instance, familiar speaker recognition is faster and more accurate when the voice is paired with a time-synchronized face from the same individual than when presented alone, and slower and less accurate when paired with the face of a different individual3.…”

mentioning

confidence: 99%

“Hearing faces and seeing voices”: Amodal coding of person identity in the human brain

Hasan

Valdés-Sosa

Groß

et al. 2016

Sci Rep

View full text Add to dashboard Cite

Recognizing familiar individuals is achieved by the brain by combining cues from several sensory modalities, including the face of a person and her voice. Here we used functional magnetic resonance (fMRI) and a whole-brain, searchlight multi-voxel pattern analysis (MVPA) to search for areas in which local fMRI patterns could result in identity classification as a function of sensory modality. We found several areas supporting face or voice stimulus classification based on fMRI responses, consistent with previous reports; the classification maps overlapped across modalities in a single area of right posterior superior temporal sulcus (pSTS). Remarkably, we also found several cortical areas, mostly located along the middle temporal gyrus, in which local fMRI patterns resulted in identity “cross-classification”: vocal identity could be classified based on fMRI responses to the faces, or the reverse, or both. These findings are suggestive of a series of cortical identity representations increasingly abstracted from the input modality.

show abstract

Hearing Faces and Seeing Voices: The Integration and Interaction of Face and Voice Processing

Cited by 37 publications

References 71 publications

Sex Differences in Facial and Vocal Attractiveness Among College Students in China

Sex Differences in Facial and Vocal Attractiveness Among College Students in China

The effect of inserting an inter-stimulus interval in face–voice matching tasks

“Hearing faces and seeing voices”: Amodal coding of person identity in the human brain

Contact Info

Product

Resources

About