Are Face-Incongruent Voices Harder to Process?

Huestegge, Sujata M.; Raettig, Tim; Huestegge, Lynn

doi:10.1027/1618-3169/a000440

Cited by 6 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Presumably, this type of conflicting information is more detrimental for naturally co‐occurring stimuli such as faces and voices than for abstract stimuli. Independent of the emotional category of the voice, responses were slower for gender‐mismatching than ‐matching face‐voice pairs, replicating previous findings (Huestegge et al, 2019; Latinus et al, 2010). Gender‐mismatching information might be more difficult to integrate due to the long‐term and repeated strengthening of the associations of gender‐congruent faces and voices.…”

Section: Discussionsupporting

confidence: 87%

Gender congruence and emotion effects in cross‐modal associative learning: Insights from ERPs and pupillary responses

Ziereis

Schacht

2023

Psychophysiology

View full text Add to dashboard Cite

Social and emotional cues from faces and voices are highly relevant and have been reliably demonstrated to attract attention involuntarily. However, there are mixed findings as to which degree associating emotional valence to faces occurs automatically. In the present study, we tested whether inherently neutral faces gain additional relevance by being conditioned with either positive, negative, or neutral vocal affect bursts. During learning, participants performed a gender‐matching task on face‐voice pairs without explicit emotion judgments of the voices. In the test session on a subsequent day, only the previously associated faces were presented and had to be categorized regarding gender. We analyzed event‐related potentials (ERPs), pupil diameter, and response times (RTs) of N = 32 subjects. Emotion effects were found in auditory ERPs and RTs during the learning session, suggesting that task‐irrelevant emotion was automatically processed. However, ERPs time‐locked to the conditioned faces were mainly modulated by the task‐relevant information, that is, the gender congruence of the face and voice, but not by emotion. Importantly, these ERP and RT effects of learned congruence were not limited to learning but extended to the test session, that is, after removing the auditory stimuli. These findings indicate successful associative learning in our paradigm, but it did not extend to the task‐irrelevant dimension of emotional relevance. Therefore, cross‐modal associations of emotional relevance may not be completely automatic, even though the emotion was processed in the voice.

show abstract

Section: Discussionsupporting

confidence: 87%

Gender congruence and emotion effects in cross‐modal associative learning: Insights from ERPs and pupillary responses

Ziereis

Schacht

2023

Psychophysiology

View full text Add to dashboard Cite

show abstract

“…Presumably, this type of conflicting information is more detrimental for naturally co-occurring stimuli such as faces and voices than for abstract stimuli. Independent of the emotional category of the voice, responses were slower for gender-mismatching than -matching face-voice pairs, replicating previous findings (Huestegge, Raettig, & Huestegge, 2019;Latinus, VanRullen, & Taylor, 2010). Gender-mismatching 10 information might be more difficult to integrate due to the long-term and repeated strengthening of the associations of gender-congruent faces and voices.…”

Section: Emotion and Congruence Effects Of Face-voice Pairssupporting

confidence: 84%

Gender Congruence and Emotion Effects in Cross-Modal Associative Learning: Insights from ERPs and Pupillary Responses

Ziereis¹,

Schacht²

2023

Preprint

View full text Add to dashboard Cite

Social and emotional cues from faces and voices are highly relevant and have been reliably demonstrated to attract attention involuntarily. However, there are mixed findings as to which degree associating emotional valence to faces occurs automatically. In the present study, we tested whether inherently neutral faces gain additional relevance by being conditioned with either positive, negative, or neutral vocal affect bursts. During learning, participants performed a gender-matching task on face-voice pairs without explicit emotion judgments of the voices. In the test session on a subsequent day, only the pre-viously associated faces were presented and had to be categorized regarding gender. We analyzed event-related potentials (ERPs), pupil diameter, and response times (RTs) of N = 32 subjects. Emotion effects were found in auditory ERPs and RTs during the learning session, suggesting that task-irrelevant emo-tion was automatically processed. However, ERPs time-locked to the conditioned faces were mainly modulated by the task-relevant information, i.e., the gender congruence of the face and voice, but not by emotion. Importantly, these ERP and RT effects of learned congruence were not limited to learning but extended to the test session, i.e., after removing the auditory stimuli. These findings indicate suc-cessful associative learning in our paradigm, but it did not extend to the task-irrelevant dimension of emotional relevance. Therefore, cross-modal associations of emotional relevance may not be completely automatic, even though the emotion was processed in the voice.

show abstract

“…In the following, one final potential limitation of the present study should be discussed in more detail. Specifically, one might argue that the present procedure (simultaneous presentation of two visual static/dynamic stimuli along with auditory speech) might represent an a priori disadvantage for the dynamic condition due to potential interference phenomena (see Huestegge and Raettig, 2019; Huestegge et al, 2019, for underlying cognitive mechanisms of such interference phenomena). In particular, different text material was used for the two input channels in the dynamic condition, resulting in an audio-visual mismatch between the lip movements of the actors and the auditory text (i.e., seen and heard speech).…”

Section: Discussionmentioning

confidence: 99%

Matching Unfamiliar Voices to Static and Dynamic Faces: No Evidence for a Dynamic Face Advantage in a Simultaneous Presentation Paradigm

Huestegge

2019

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

Previous research has demonstrated that humans are able to match unfamiliar voices to corresponding faces and vice versa. It has been suggested that this matching ability might be based on common underlying factors that have a characteristic impact on both faces and voices. Some researchers have additionally assumed that dynamic facial information might be especially relevant to successfully match faces to voices. In the present study, static and dynamic face-voice matching ability was compared in a simultaneous presentation paradigm. Additionally, a procedure (matching additionally supported by incidental association learning) was implemented which allowed for reliably excluding participants that did not pay sufficient attention to the task. A comparison of performance between static and dynamic face-voice matching suggested a lack of substantial differences in matching ability, suggesting that dynamic (as opposed to mere static) facial information does not contribute meaningfully to face-voice matching performance. Importantly, this conclusion was not merely derived from the lack of a statistically significant group difference in matching performance (which could principally be explained by assuming low statistical power), but from a Bayesian analysis as well as from an analysis of the 95% confidence interval (CI) of the actual effect size. The extreme border of this CI suggested a maximally plausible dynamic face advantage of less than four percentage points, which was considered way too low to indicate any theoretically meaningful dynamic face advantage. Implications regarding the underlying mechanisms of face-voice matching are discussed.

show abstract

Are Face-Incongruent Voices Harder to Process?

Cited by 6 publications

References 42 publications

Gender congruence and emotion effects in cross‐modal associative learning: Insights from ERPs and pupillary responses

Gender congruence and emotion effects in cross‐modal associative learning: Insights from ERPs and pupillary responses

Gender Congruence and Emotion Effects in Cross-Modal Associative Learning: Insights from ERPs and Pupillary Responses

Matching Unfamiliar Voices to Static and Dynamic Faces: No Evidence for a Dynamic Face Advantage in a Simultaneous Presentation Paradigm

Contact Info

Product

Resources

About