Effects of Audiovisual Interactions on Working Memory Task Performance—Interference or Facilitation

Abstract

(1) Background: The combined n-back + Go/NoGo paradigm was used to investigate whether audiovisual interactions interfere with or facilitate WM. (2) Methods: College students were randomly assigned to perform the working memory task based on either a single (visual or auditory) or dual (audiovisual) stimulus. Reaction times, accuracy, and WM performance were compared across the two groups to investigate effects of audiovisual interactions. (3) Results: With low cognitive load (2-back), auditory stimuli had no effect on visual working memory, whereas visual stimuli had a small effect on auditory working memory. With high cognitive load (3-back), auditory stimuli interfered (large effect size) with visual WM, and visual stimuli interfered (medium effect size) with auditory WM. (4) Conclusions: Audiovisual effects on WM follow the resource competition theory, and the cognitive load of a visual stimulus is dominated by competition; vision always interferes with audition, and audition conditionally interferes with vision. With increased visual cognitive load, competitive effects of audiovisual interactions were more obvious than those with auditory stimuli. Compared with visual stimuli, audiovisual stimuli showed significant interference only when visual cognitive load was high. With low visual cognitive load, the two stimulus components neither facilitated nor interfered with the other in accordance with a speed–accuracy trade-off.

Audiovisual n-Back Training Alters the Neural Processes of Working Memory and Audiovisual Integration: Evidence of Changes in ERPs

“…Therefore, we inferred that better training effects in the 2-and 3-back conditions were related to increased audiovisual benefits under higher cognitive loads. Notably, we recruited college students who may possess better cognitive abilities than the general population [40]. In the 1-back condition, a ceiling effect may exist, resulting in no increases in training gains in this condition.…”

The Contribution of Visual and Auditory Working Memory and Non-Verbal IQ to Motor Multisensory Processing in Elementary School Children

“…Indeed, in adults, there is evidence that the improvements on multisensory MRT and serial reaction time (SRT) tasks are linked to the visual WM capacity [2,81,82]. However, there is relatively little child research examining the specific associations between either visual or auditory WM and multisensory MRTs, with Barutchu and colleagues [3,12] also failing to find a systematic correlation between measures of multisensory MRTs and auditory WM.…”

Effects of audiovisual interactions on working memory: Use of the combined N-back + Go/NoGo paradigm

“…The brain may need to modulate and integrate information from both vision and hearing to reach judgments about higher cognitive functions such as consciousness and behavior (Tang et al, 2016). Interestingly, the results of a recent behavioral experiment by Yang et al that combined audiovisual interaction with a WM task suggest that audiovisual interaction may be influenced by cognitive load (He et al, 2022); specifically, under high cognitive load, there is substantial audiovisual interference, whereas under low cognitive load, there is no detectable interference or facilitation. These findings are consistent with a trade-off between speed and accuracy.…”