No abstract

Much debate revolves around how the cortex integrates information from different sensory modalities to generate predictions about sensory events. The classic perspective that multisensory processes are organised along a hierarchy from strictly modality-specific to supramodal processes has been challenged by evidence suggesting that the cortex is multisensory at all levels. Here we examined whether responses to sensory expectation violations which require the integration of inputs from different sensory modalities rely on early or late cortical functions. To that end, we designed a crossmodal hierarchical oddball paradigm in which expectations about sensory inputs were generated and violated in different sensory modalities at two nested hierarchical levels. We investigated whether there are shared cortical signatures between the crossmodal effect and either lower-order unimodal local responses or higher-order global responses to sensory surprise thought to be modality-independent. A temporal decoding analysis revealed that crossmodal effects unfold at different stages of the cortical hierarchy. Using temporal decoding, we found that the crossmodal effect shared cortical dynamics with its unimodal counterpart but not with global responses to sensory expectation violations. Beyond shared dynamics, we also demonstrate that cross- and unimodal prediction violations modulate common connections in a network with interconnected modality-specific early and late frontal regions.

Pre-sleep exposure to short-wavelength light suppresses melatonin and decreases sleepiness with activating effects extending to sleep. This has mainly been attributed to melanopic effects, but mechanistic insights are missing. Thus, we investigated whether two light conditions only differing in the melanopic effects (123 vs. 59 lux melanopic EDI) differentially affect sleep besides melatonin. Additionally, we studied whether the light differentially modulates sensory processing during wakefulness and sleep. Twenty-nine healthy volunteers (18-30 years, 15 women) were exposed to two metameric light conditions (high- vs. low-melanopic, ≈60 photopic lux) for 1 hour ending 50 min prior to habitual bed time. This was followed by an 8-h sleep opportunity with polysomnography. Objective sleep measurements were complemented by self-report. Salivary melatonin, subjective sleepiness, and behavioural vigilance were sampled at regular intervals. Sensory processing was evaluated during light exposure and sleep on the basis of neural responses related to violations of expectations in an oddball paradigm. We observed suppression of melatonin by ≈14 % in the high- compared to the low-melanopic condition. However, conditions did not differentially affect sleep, sleep quality, sleepiness, or vigilance. A neural mismatch response was evident during all sleep stages, but not differentially modulated by light. Suppression of melatonin by light targeting the melanopic system does not automatically translate to acutely altered levels of vigilance or sleepiness or to changes in sleep, sleep quality, or basic sensory processing. Given contradicting earlier findings and the retinal anatomy, this may suggest that an interaction between melanopsin and cone-rod signals needs to be considered.

To ensure survival in a dynamic environment, the human neocortex monitors input streams from different sensory organs for important sensory events. Which principles govern whether different senses share common or modality-specific brain networks for sensory target detection? We examined whether complex targets evoke sustained supramodal activity while simple targets rely on modality-specific networks with short-lived supramodal contributions. In a series of hierarchical multisensory target detection studies (n= 77, of either sex) using EEG, we applied a temporal cross-decoding approach to dissociate supramodal and modality-specific cortical dynamics elicited by rule-based global and feature-based local sensory deviations within and between the visual, somatosensory, and auditory modality. Our data show that each sense implements a cortical hierarchy orchestrating supramodal target detection responses, which operate at local and global timescales in successive processing stages. Across different sensory modalities, simple feature-based sensory deviations presented in temporal vicinity to a monotonous input stream triggered a mismatch negativity-like local signal which decayed quickly and early, whereas complex rule-based targets tracked across time evoked a P3b-like global neural response which generalized across a late time window. Converging results from temporal cross-modality decoding analyses across different datasets, we reveal that global neural responses are sustained in a supramodal higher-order network, whereas local neural responses canonically thought to rely on modality-specific regions evolve into short-lived supramodal activity. Together, our findings demonstrate that cortical organization largely follows a gradient in which short-lived modality-specific as well as supramodal processes dominate local responses, whereas higher-order processes encode temporally extended abstract supramodal information fed forward from modality-specific cortices.SIGNIFICANCE STATEMENTEach sense supports a cortical hierarchy of processes tracking deviant sensory events at multiple timescales. Conflicting evidence produced a lively debate around which of these processes are supramodal. Here, we manipulated the temporal complexity of auditory, tactile, and visual targets to determine whether cortical local and global ERP responses to sensory targets share cortical dynamics between the senses. Using temporal cross-decoding, we found that temporally complex targets elicit a supramodal sustained response. Conversely, local responses to temporally confined targets typically considered modality-specific rely on early short-lived supramodal activation. Our finding provides evidence for a supramodal gradient supporting sensory target detection in the cortex, with implications for multiple fields in which these responses are studied (e.g., predictive coding, consciousness, and attention).