Emotional valence and spatial congruency differentially modulate crossmodal processing: an fMRI study

Abstract

Salient exogenous stimuli modulate attentional processes and lead to attention shifts–even across modalities and at a pre-attentive level. Stimulus properties such as hemispheric laterality and emotional valence influence processing, but their specific interaction in audio-visual attention paradigms remains ambiguous. We conducted an fMRI experiment to investigate the interaction of supramodal spatial congruency, emotional salience, and stimulus presentation side on neural processes of attention modulation. Emotionally neutral auditory deviants were presented in a dichotic listening oddball design. Simultaneously, visual target stimuli (schematic faces) were presented, which displayed either a negative or a positive emotion. These targets were presented in the left or in the right visual field and were either spatially congruent (valid) or incongruent (invalid) with the concurrent deviant auditory stimuli. According to our expectation we observed that deviant stimuli serve as attention-directing cues for visual target stimuli. Region-of-interest (ROI) analyses suggested differential effects of stimulus valence and spatial presentation on the hemodynamic response in bilateral auditory cortices. These results underline the importance of valence and presentation side for attention guidance by deviant sound events and may hint at a hemispheric specialization for valence and attention processing.

Interpretation of Social Interactions: Functional Imaging of Cognitive-Semiotic Categories During Naturalistic Viewing

“…As expected, the bilateral STG responded most strongly to verbal interactions, whereas non-verbal interactions activated visual pathways. The latter finding suggests that during verbal stimulation visual cues are less processed ( Wolf et al, 2014 ). In addition to this clear pattern, the non-verbal, non-auditory interactions yielded a cross-modal modulation in the bilateral STG extending into the MTG.…”

Neural processing of emotion in multimodal settings

“…Moreover, sweat from social anxiety situations enhanced the processing of fearful facial stimuli only in socially anxious individuals—an impressive example of the integration of fear-relevant cues being influenced by personality traits. The interaction of visual emotion processing with irrelevant auditory cues was also subject of the study by Wolf et al ( 2014 ). The authors demonstrated that visual emotion cues modulated tone processing in the auditory cortex.…”

Prosody production networks are modulated by sensory cues and social context

“…Previous research has shown that visual processing pathways reach far down into the STG, making it a central hub for the multimodal integration of auditory and visual signals ( Robins et al , 2009 ; Klasen et al , 2011 ). Visual cues can influence responses in auditory cortex ( Harrison and Woodhouse, 2016 ) and facilitate processing in the auditory domain ( Wolf et al , 2014 ). Moreover, the STG supports auditory but not visual sensory imagination ( Zvyagintsev et al , 2013 ).…”