Neural Basis of Semantically Dependent and Independent Cross-Modal Boosts on the Attentional Blink

Abstract

The present study recorded event-related potentials (ERPs) in a visual object-recognition task under the attentional blink paradigm to explore the temporal dynamics of the cross-modal boost on attentional blink and whether this auditory benefit would be modulated by semantic congruency between T2 and the simultaneous sound. Behaviorally, the present study showed that not only a semantically congruent but also a semantically incongruent sound improved T2 discrimination during the attentional blink interval, whereas the enhancement was larger for the congruent sound. The ERP results revealed that the behavioral improvements induced by both the semantically congruent and incongruent sounds were closely associated with an early cross-modal interaction on the occipital N195 (192–228 ms). In contrast, the lower T2 accuracy for the incongruent than congruent condition was accompanied by a larger late occurring cento-parietal N440 (424–448 ms). These findings suggest that the cross-modal boost on attentional blink is hierarchical: the task-irrelevant but simultaneous sound, irrespective of its semantic relevance, firstly enables T2 to escape the attentional blink via cross-modally strengthening the early stage of visual object-recognition processing, whereas the semantic conflict of the sound begins to interfere with visual awareness only at a later stage when the representation of visual object is extracted.

The interplay between audiovisual temporal synchrony and semantic congruency in the cross‐modal boost of the visual target discrimination during the attentional blink

“…Furthermore, the neural generators of unisensoryevoked N400 effects have been localized mainly to the middle/ superior temporal cortex (Khateb et al, 2007;Khateb, Pegna, Landis, Mouthon, & Annoni, 2010;Lau, Phillips, & Poeppel, 2008), which is also a well-documented brain region of multisensory integration (Beauchamp, 2005;Calvert, 2001). The statistical analysis yielded that the present cross-modal N440 component was elicited significantly only when a semantically incongruent sound synchronized with T2, in accordance with the recent finding that this component was sensitive only to audiovisual stimuli with semantic conflict (Zhao et al, 2021).…”

“…Interestingly, the current study showed that the occipitally distributed early cross-modal ERP component (i.e., P195) was positive in polarity, whereas this component was found to be negative (i.e., N195) in the recent study conducted by Zhao et al (2021).…”

“…The electrodes FCz and were where auditory-evoked ERP components (P1, N1 and P2) and visual-evoked ERP components (P1 and N1) were largest, respectively, whereas the electrodes P1 and O1 were where the crucial cross-modal N440 (424-448 ms) and P195 (192-228 ms) components were quantified, respectively congruent or incongruent sound preceded T2, synchronized with T2, and was delayed relative to T2, were isolated respectively by calculating cross-modal difference waveforms (see section 2.4). The occipitally distributed early cross-modal negativity N195 (192-228 ms after T2 onset at the electrode O1), which has been shown to underlie the cross-modal boost during the attentional blink(Zhao et al, 2021), was manifested as a positive component in the present study (Figure4b).Thus, this component was labeled as P195 in the present study for clarity. Then, a 3 (sound temporal position: precede, synchronize, delay) Â 2 (semantic congruency: congruent, incongruent) repeatedmeasures ANOVA was conducted on the mean amplitudes of P195 component.…”

“…These line drawings and natural sounds were all taken from the study of Zhao et al (2021) wherein a similar basic experimental design was used.…”

Are auditory cues special? Evidence from cross-modal distractor-induced blindness

“…This finding contradicts the assumption of a generally larger effect of visual compared to auditory distractors on the detection of a target in the same modality. That the cross-modal distractor effect appears to be reduced compared to the unimodal visual effect might be attributed to an enhancement of visual target processing by a concurrent auditory cue, which has been demonstrated for other paradigms (Frassinetti et al, 2002;Noesselt et al, 2008;Noesselt et al, 2010;Van der Burg et al, 2011;Zhao et al, 2020). This multisensory enhancement might counteract the distractor-induced inhibition to a certain degree.…”