Semantic Congruency Modulates the Effect of Attentional Load on the Audiovisual Integration of Animate Images and Sounds

Li, Qingqing; Wu, Qiong; Yu, Yiyang; Wu, Fengxia; Takahashi, Satoshi; Ejima, Yoshimichi; Yang, Jiajia; Wu, Jinglong

doi:10.1177/2041669520981096

Cited by 4 publications

(2 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies have shown that incongruent stimuli have an effect on the modulation of multisensory integration ( Guo et al, 2015 ; Laurienti et al, 2004 ). Li et al (2020) used unisensory stimuli (animal images or sounds) and multisensory stimuli (semantically congruent audiovisual objects or semantically incongruent audiovisual objects) to assess multisensory integration. Results indicated that semantically incongruent animal sounds and images were not integrated ( Li et al, 2020 ).…”

Section: Discussionmentioning

confidence: 99%

“… Li et al (2020) used unisensory stimuli (animal images or sounds) and multisensory stimuli (semantically congruent audiovisual objects or semantically incongruent audiovisual objects) to assess multisensory integration. Results indicated that semantically incongruent animal sounds and images were not integrated ( Li et al, 2020 ). More than that, the incongruent flankers conflicted with not only target arrow but also auditory stimuli in Experiment 2.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The Effects of Bilateral and Ipsilateral Auditory Stimuli on the Subcomponents of Visual Attention

Guo

Tang

et al. 2021

i-Perception

View full text Add to dashboard Cite

Attention contains three functional network subcomponents of alerting, orienting, and executive control. The attention network test (ANT) is usually used to measure the efficiency of three attention subcomponents. Previous researches have focused on examining the unimodal attention with visual or auditory ANT paradigms. However, it is still unclear how an auditory stimulus influences the visual attention networks. This study investigated the effects of bilateral auditory stimuli (Experiment 1) and ipsilateral auditory stimulus (Experiment 2) on the visual attention subcomponents. We employed an ANT paradigm and manipulated the target modality types, including visual and audiovisual modalities. The participants were instructed to distinguish the direction of the central arrow surrounded by distractor arrows. In Experiment 1, we found that the simultaneous bilateral auditory stimuli reduced the efficiency of visual alerting and orienting, but had no significant effect on the efficiency of visual executive control. In Experiment 2, the ipsilateral auditory stimulus reduced the efficiency of visual executive control, but had no significant effect on the efficiency of visual alerting and orienting. We also observed a reduced relative multisensory response enhancement (rMRE) effect in cue condition relative to no cue condition (Experiment 1), and an increased rMRE effect in congruent condition compared with incongruent condition (Experiment 2). These results firstly provide evidence for the alerting, orienting and executive control effects in audiovisual condition. And the bilateral and ipsilateral auditory stimuli have different effects on the subcomponents of visual attention.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

The Effects of Bilateral and Ipsilateral Auditory Stimuli on the Subcomponents of Visual Attention

Guo

Tang

et al. 2021

i-Perception

View full text Add to dashboard Cite

show abstract

The impact of environmental sounds on food reward

Peng-Li

Andersen

Finlayson

et al. 2022

Physiology & Behavior

View full text Add to dashboard Cite

Semantic congruency modulates the speed-up of multisensory responses

Roberts,

Jentzsch,

Otto

2024

Sci Rep

View full text Add to dashboard Cite

Responses to multisensory signals are often faster compared to their unisensory components. This speed-up is typically attributed to target redundancy in that a correct response can be triggered by one or the other signal. In addition, semantic congruency of signals can also modulate multisensory responses; however, the contribution of semantic content is difficult to isolate as its manipulation commonly changes signal redundancy as well. To disentangle the effects of redundancy and semantic congruency, we manipulated semantic content but kept redundancy constant. We presented semantically congruent/incongruent animal pictures and sounds and asked participants to respond with the same response to two target animals (cats and dogs). We find that the speed-up of multisensory responses is larger for congruent (e.g., barking dogs) than incongruent combinations (e.g., barking cats). We then used a computational modelling approach to analyse audio-visual processing interferences that may underlie the effect. Our data is best described by a model that explains the semantic congruency modulation with a parameter that was previously linked to trial sequence effects, which in our experiment occur from the repetition/switching of both sensory modality and animal category. Yet, a systematic analysis of such trial sequence effects shows that the reported congruency effect is an independent phenomenon. Consequently, we discuss potential contributors to the semantic modulation of multisensory responses.

show abstract

Semantic Congruency Modulates the Effect of Attentional Load on the Audiovisual Integration of Animate Images and Sounds

Cited by 4 publications

References 67 publications

The Effects of Bilateral and Ipsilateral Auditory Stimuli on the Subcomponents of Visual Attention

The Effects of Bilateral and Ipsilateral Auditory Stimuli on the Subcomponents of Visual Attention

The impact of environmental sounds on food reward

Semantic congruency modulates the speed-up of multisensory responses

Contact Info

Product

Resources

About