Visual cortex responds to transient sound changes without encoding complex auditory dynamics

Abstract

Signals encoded in one sensory modality enhance cortical sensitivity for co-occurring signals in another modality. Previous research has demonstrated that this facilitation occurs through crossmodal modulations of cortical oscillatory activity. However, the neural origin of these signals and auditory information conveyed by this mechanism remain poorly understood. Using intracranial electroencephalography (iEEG) in humans, we examined the temporal dynamics of crossmodal modulations to identify the level of hierarchical coding at which auditory information is transmitted to visual cortex. Subcortical auditory neurons exhibit frequency following behaviors in response to amplitude-modulated sounds, with oscillatory activity entrained at the rhythmic rate of the auditory signal. This auditory response is paralleled in the visual system by the entrainment of visual neurons to the rhythmic rate of flashing strobe lights. In contrast, ~20% of neurons in auditory cortex do not entrain to amplitude modulations but respond only to the onsets and/or offsets of auditory stimuli. In visual cortex, amplitude-modulated sounds elicited transient onset and offset responses in multiple areas, but no entrainment to the sounds’ modulations frequencies. These results suggest that auditory information conveyed to visual cortex does not include temporally fine-grained stimulus dynamics encoded by the auditory midbrain and thalamus but, rather, a temporally segmented representation of auditory events that emerges only in auditory cortex. Crossmodal responses were maximal in low-level visual cortex, potentially implicating a direct pathway for rapid interactions between low-to-mid-level auditory and visual cortices. This mechanism may facilitate perception by time-locking visual computations to environmental events marked by discontinuities in auditory input.