Audio-visual synchrony and feature-selective attention co-amplify early visual processing

Abstract

2 ABSTRACT Our brain relies on neural mechanisms of selective attention and converging sensory processing to efficiently cope with rich and unceasing multisensory inputs. One prominent assumption holds that audio-visual synchrony can act as a strong attractor for spatial attention. Here, we tested for a similar effect of audio-visual synchrony on feature-selective attention. We presented two superimposedGabor patches that differed in colour and orientation. On each trial participants were cued to selectively attend to one of the two patches. Over time, spatial frequencies of both patches varied sinusoidally at distinct rates (3.14 and 3.63 Hz) giving rise to pulse-like percepts. A simultaneously presented pure tone carried a frequency modulation at the pulse rate of one of the two visual stimuli to introduce audio-visual synchrony. Pulsed stimulation elicited distinct time-locked oscillatory electrophysiological brain responses. These steady-state responses (SSRs) were quantified in the spectral domain to examine individual stimulus processing under conditions of synchronous vs.asynchronous tone presentation and when respective stimuli were attended vs. unattended. We found that both, attending to the colour of a stimulus and its synchrony with the tone, enhanced its processing. Moreover, both gain effects combined linearly for attended in-sync stimuli. Our results suggest that audio-visual synchrony can attract attention to specific stimulus features when stimuli overlap in space.

Audio-visual synchrony and spatial attention enhance processing of dynamic visual stimulation independently and in parallel: a frequency-tagging study

“…SSR power decreased with increasing stimulus presentation rate (main effect SSR component : F(2,22) = 55.76, P GG < 0.001, ε GG = 0.90, η 2 = 0.301; also see Figure 3 ) as has been documented extensively before (Keitel and Müller, 2015; Porcu et al, 2014). Figure 3c underlines that amplitudes further varied with the allocation of attention towards stimuli (main effect attention : F(1,11) = 24.15, P < 0.001, η 2 = 0.094) and were affected by audio-visual synchrony (F(1,11) = 71.01, P < 0.001, η 2 = 0.067).…”

“…In contrast to attentional gain, results of an earlier investigation suggested that synchrony-related gain effects may be specific to pulsation-driven SSRs. Using a paradigm similar to the present study, Keitel and Müller (2015) found that an SSR component with a frequency of twice the pulsation rate was exclusively susceptible to synchrony-related gain effects. At this rate, the stimulation presumably contained strong transients critical for establishing audio-visual synchrony (Werner and Noppeney, 2011).…”

“…Data of 12 participants aged 18 – 31 years (all right-handed, 9 female) entered analyses. Previous studies have used comparable sample sizes to reliably (re)produce effects of spatial attention (Ding, 2005; Müller et al, 1998a; 1998b; Walter et al, 2015; Zhang et al, 2010) and audio-visual synchrony (Jenkins et al, 2011; Keitel and Müller, 2015; Nozaradan et al, 2012) on SSRs.…”

“…Prior to the experiment, we employed the method of limits (Leek, 2001) to approximate individual hearing thresholds using one of the experimental stimuli, a 3.14-Hz frequency modulated tone (see e.g. Herrmann et al, 2014; Keitel and Müller, 2015). In our implementation, participants listened to a series of 10 tone sequences with a maximum duration of 15 s per sequence.…”

“…Notably, spectra revealed strong harmonic responses at twice the pulse frequencies (6.28 and 7.24 Hz). We included these pulse-driven harmonics in further analyses because fundamental and harmonic responses have been hypothesized to reflect different aspects of stimulus processing (Kim et al, 2011; Pastor et al, 2007; Porcu et al, 2013) and showed modulation by synchrony in a previous study (Keitel and Müller, 2015). Grand-average topographical distribution of pulse-driven as well as flicker-driven SSR power averaged over conditions showed widespread maxima at parieto-occipital electrode sites (scalp maps in Figure 2a ) that are typically observed in experiments with lateralized flicker stimulation (see e.g.…”