Sensory target detection at local and global timescales reveals a hierarchy of supramodal dynamics in the human cortex

Abstract

Much debate revolves around how the cortex integrates information from different sensory modalities to generate predictions about sensory events. The classic perspective that multisensory processes are organised along a hierarchy from strictly modality-specific to supramodal processes has been challenged by evidence suggesting that the cortex is multisensory at all levels. Here we examined whether responses to sensory expectation violations which require the integration of inputs from different sensory modalities rely on early or late cortical functions. To that end, we designed a crossmodal hierarchical oddball paradigm in which expectations about sensory inputs were generated and violated in different sensory modalities at two nested hierarchical levels. We investigated whether there are shared cortical signatures between the crossmodal effect and either lower-order unimodal local responses or higher-order global responses to sensory surprise thought to be modality-independent. A temporal decoding analysis revealed that crossmodal effects unfold at different stages of the cortical hierarchy. Using temporal decoding, we found that the crossmodal effect shared cortical dynamics with its unimodal counterpart but not with global responses to sensory expectation violations. Beyond shared dynamics, we also demonstrate that cross- and unimodal prediction violations modulate common connections in a network with interconnected modality-specific early and late frontal regions.

Supramodal neural information supports stimulus-driven attention across cortical levels

“…With these results, we replicated and extended earlier findings that deviations of sensory stimulus groups elicit a P3b-like difference wave from ˜200 ms until trial end in either sensory modality [5,48]. We also provide corroborating evidence for our own finding that shared sustained representations between the auditory and the somatosensory modality emerge from ˜200 ms in both the salient and regular contrast [5].…”

“…Cortical responses to sensory targets manifest in a late and sustained supramodal activation pattern regardless of targets being regular or salient. Characteristics of this response are consistent with the P3b, an attention-related positive difference wave which occurs ˜300 ms after a deviant stimulus [48] and has been found to rely on a supramodal network in earlier studies [5,47]. While regular deviations only trigger a P3b-like temporally extended activity pattern, salient deviations involve a supramodal serial mid-latency process which evolves into sustained activity from ˜300 ms.…”

“…Building up on previous studies showing that sensory deviations elicit activity in supramodal brain areas [5,46,47], we investigated whether salient and regular targets lead to different supramodal dynamics in the cortex. Temporal decoding within and between sensory modalities revealed differences in the cortical processing of regular and salient deviations (Figure 3, A).…”

“…Attention is a fundamental cognitive function used to select important sensory events for prioritised processing [1]. To monitor the sensory environment for such events, each perceptual modality implements an attention-controlled system for sensory target detection spanning from primary modality-specific to frontal supramodal regions [2][3][4][5].…”

“…Attention is a fundamental cognitive function used to select important sensory events for prioritised processing [1]. To monitor the sensory environment for such events, each perceptual modality implements an attention-controlled system for sensory target detection spanning from primary modality-specific to frontal supramodal regions [2–5]. A function of sensory target detection systems is to recognize salient sensory events with specific physical features which cause the stimulus to perceptually “pop out” of the background, grabbing attention in a bottom-up process [6].…”