Coherent emotional perception from body expressions and the voice

Abstract

Perceiving emotion from multiple modalities enhances the perceptual sensitivity of an individual. This allows more accurate judgments of others' emotional states, which is crucial to appropriate social interactions. It is known that body expressions effectively convey emotional messages, although fewer studies have examined how this information is combined with the auditory cues. The present study used event-related potentials (ERP) to investigate the interaction between emotional body expressions and vocalizations. We also examined emotional congruency between auditory and visual information to determine how preceding visual context influences later auditory processing. Consistent with prior findings, a reduced N1 amplitude was observed in the audiovisual condition compared to an auditory-only condition. While this component was not sensitive to the modality congruency, the P2 was sensitive to the emotionally incompatible audiovisual pairs. Further, the direction of these congruency effects was different in terms of facilitation or suppression based on the preceding contexts. Overall, the results indicate a functionally dissociated mechanism underlying two stages of emotional processing whereby N1 is involved in cross-modal processing, whereas P2 is related to assessing a unifying perceptual content. These data also indicate that emotion integration can be affected by the specific emotion that is presented.

Dynamic Facial Expressions Prime the Processing of Emotional Prosody

“…In contrast to audiovisual emotion studies that reported emotional congruency effects also in the P200 (Pourtois et al, 2000 ; Ho et al, 2015 ; Kokinous et al, 2015 ; Zinchenko et al, 2015 ) or exclusively in the P200 (Balconi and Carrera, 2011 ; Yeh et al, 2016 ; Zinchenko et al, 2017 ), we failed to observe any significant ERP differences for this component. We argue that in studies showing congruency effects only in the P200, different mechanisms may have shifted emotional congruency effects toward the P200: Balconi and Carrera ( 2011 ) used static facial displays whose onset was temporally aligned to their prosodic stimuli; therefore participants may have needed longer than in other studies for combining auditory and visual cues.…”

“…They found a P200 amplitude reduction for audiovisual compared to unimodal stimulation, but no N100 effects. Yeh et al ( 2016 ) used bodily expressions, which may be a less reliable predictor of vocal emotional expressions than a face (although these authors did show N100 suppression during audiovisual compared to auditory processing, but irrespective of congruency). Furthermore, identity mismatches between the visual and auditory tracks could have played a role in their study, because the materials came from different stimulus databases.…”

Altered neuronal habituation to hearing others’ pain in adults with autistic traits

“…Consistent with the results of a previous study 8 , in the present study, the P2 amplitudes of S1 in response to the painful voices in the Passive Listening Task were smaller than in the Pain Judgment Task. Since the P2 response is relevant to the emotional quality of the audio recordings 31 , and since our previous studies showed that the P2 amplitudes related to painful voices were significantly and positively correlated with pain intensity ratings 8,17 , P2 amplitudes to painful voices at least partly reflected cognitive empathy for others' vocal pain 8 . This suggests that, compared with the Passive Listening Task, when focusing on others' vocal pain in the Pain Judgement Task, participants showed more empathic neuronal responses to others' pain.…”

Teachers’ Beliefs About Children’s Anger and Skill in Recognizing Children’s Anger Expressions

“…We also acknowledge that teachers in our study were tasked with identifying only facially expressed anger. Clearly, vocal, bodily, behavioral, situational, and physiological cues provide important and distinct emotional information that allow for holistic judgments about emotions (Yeh et al, 2016), especially for anger (Cacioppo et al, 2000). In fact, these different modalities of emotion expression may be particularly important cues for some emotions (Schirmer and Adolphs, 2017).…”