Memory Load Alters Perception-Related Neural Oscillations during Multisensory Integration

Abstract

Integrating information across different senses is a central feature of human perception. Previous research suggests that multisensory integration is shaped by a context-dependent and largely adaptive interplay between stimulus-driven bottom-up and top-down endogenous influences. One critical question concerns the extent to which this interplay is sensitive to the amount of available cognitive resources. In the present study, we investigated the influence of limited cognitive resources on audiovisual integration by measuring high-density electroencephalography (EEG) in healthy participants performing the soundinduced flash illusion (SIFI) and a verbal n-back task (0-back, low load and 2-back, high load) in a dual-task design. In the SIFI, the integration of a flash with two rapid beeps can induce the illusory perception of two flashes. We found that high compared with low load increased illusion susceptibility and modulated neural oscillations underlying illusion-related crossmodal interactions. Illusion perception under high load was associated with reduced early b power (18-26 Hz, ;70 ms) in auditory and motor areas, presumably reflecting an early mismatch signal and subsequent top-down influences including increased frontal h power (7-9 Hz, ;120 ms) in mid-anterior cingulate cortex (ACC) and a later b power suppression (13-22 Hz, ;350 ms) in prefrontal and auditory cortex. Our study demonstrates that integrative crossmodal interactions underlying the SIFI are sensitive to the amount of available cognitive resources and that multisensory integration engages top-down h and b oscillations when cognitive resources are scarce.

Age influences audiovisual speech processing in multi-talker scenarios – Evidence from cortical oscillations

“…This difference was only present in the older group. In audiovisual stimuli, an increase in theta power has been linked to incongruence processing and the need for cognitive control to successfully enable audiovisual integration (Michail et al, 2021 using the sound induced flash illusion; Morís Fernández et al, 2015 using McGurk stimuli). At the time of sound onset, the preceding visual information is already partially processed.…”

“…Midfrontal theta (4-8 Hz) is thought to reflect communication between different brain areas when the need for cognitive control arises (Cavanagh & Frank, 2014). An increase in theta power can be observed in tasks with high working memory load (Gevins, 1997;Jensen & Tesche, 2002;Maurer et al, 2015; for audiovisual speech stimuli Michail et al, 2021). In audiovisual tasks, theta band activity has been attributed to divided attention (Keller et al, 2017) and the encoding of new information and change detection (Fingelkurts et al, 2007).…”

“…A decrease in beta power can be observed following stimulus-driven stimulation (Engel & Fries, 2010), and is stronger in audiovisual conditions with background noise (Schepers et al, 2013) as well as following McGurk illusions (Roa Romero et al, 2015). In audiovisual speech stimuli using McGurk illusions, an early and a late beta power suppression can be distinguished (Michail et al, 2021; Roa Romero et al, 2015). While the early suppression is interpreted as the formation of a fusion percept in McGurk illusions and error processing in incongruent audiovisual stimuli, late beta suppression is interpreted as a marker for top-down audiovisual integration (Michail et al, 2021).…”

“…In audiovisual speech stimuli using McGurk illusions, an early and a late beta power suppression can be distinguished (Michail et al, 2021; Roa Romero et al, 2015). While the early suppression is interpreted as the formation of a fusion percept in McGurk illusions and error processing in incongruent audiovisual stimuli, late beta suppression is interpreted as a marker for top-down audiovisual integration (Michail et al, 2021). Beta oscillations have been associated with networks related to executive function, with a decrease in power in older adults, together with a posterior-to-anterior shift of activation (Enriquez-Geppert & Barceló, 2018).…”

Visual Perceptual Load Attenuates Age-Related Audiovisual Integration in an Audiovisual Discrimination Task

“…Therefore, more energy was allocated to the visual stimulus under medium visual perceptual-load conditions than that under low visual perceptual-load conditions, and more under high visual perceptual-load conditions than that under medium visual perceptual-load conditions. Additionally, a recent study reported that AVI was sensitive to the amount of available cognitive resources ( Michail et al, 2021 ). In the current study, the memory loads were different among perceptual-load conditions, showing the highest memory load under high perceptual-load conditions (three attributes) and the lowest memory load under low perceptual-load conditions (one attribute).…”