Sensory Target Detection at Local and Global Timescales Reveals a Hierarchy of Supramodal Dynamics in the Human Cortex

Abstract

To ensure survival in a dynamic environment, the human neocortex monitors input streams from different sensory organs for important sensory events. Which principles govern whether different senses share common or modality-specific brain networks for sensory target detection? We examined whether complex targets evoke sustained supramodal activity while simple targets rely on modality-specific networks with short-lived supramodal contributions. In a series of hierarchical multisensory target detection studies (n= 77, of either sex) using EEG, we applied a temporal cross-decoding approach to dissociate supramodal and modality-specific cortical dynamics elicited by rule-based global and feature-based local sensory deviations within and between the visual, somatosensory, and auditory modality. Our data show that each sense implements a cortical hierarchy orchestrating supramodal target detection responses, which operate at local and global timescales in successive processing stages. Across different sensory modalities, simple feature-based sensory deviations presented in temporal vicinity to a monotonous input stream triggered a mismatch negativity-like local signal which decayed quickly and early, whereas complex rule-based targets tracked across time evoked a P3b-like global neural response which generalized across a late time window. Converging results from temporal cross-modality decoding analyses across different datasets, we reveal that global neural responses are sustained in a supramodal higher-order network, whereas local neural responses canonically thought to rely on modality-specific regions evolve into short-lived supramodal activity. Together, our findings demonstrate that cortical organization largely follows a gradient in which short-lived modality-specific as well as supramodal processes dominate local responses, whereas higher-order processes encode temporally extended abstract supramodal information fed forward from modality-specific cortices.SIGNIFICANCE STATEMENTEach sense supports a cortical hierarchy of processes tracking deviant sensory events at multiple timescales. Conflicting evidence produced a lively debate around which of these processes are supramodal. Here, we manipulated the temporal complexity of auditory, tactile, and visual targets to determine whether cortical local and global ERP responses to sensory targets share cortical dynamics between the senses. Using temporal cross-decoding, we found that temporally complex targets elicit a supramodal sustained response. Conversely, local responses to temporally confined targets typically considered modality-specific rely on early short-lived supramodal activation. Our finding provides evidence for a supramodal gradient supporting sensory target detection in the cortex, with implications for multiple fields in which these responses are studied (e.g., predictive coding, consciousness, and attention).

A multimodal cortical network of sensory expectation violation revealed by fMRI

“…We showed that early and late MMRs in our EEG data were best explained by a Bayesian observer tracking stimulus transition probabilities and that more central and later responses around the P3 appeared to specifically track stimulus transitions across multiple modalities. In recent MMR research, the local–global paradigm has revealed that the early MMN and the later P3 reflect two hierarchical stages of mismatch processing signaling regularity violation on increasing levels of sequence complexity (Bekinschtein et al, 2009 ; Chennu et al, 2013 ; Chennu et al, 2016 ; Dehaene et al, 2015 ; Dürschmid et al, 2016 ; El Karoui et al, 2015 ; King et al, 2014 ; Niedernhuber et al, 2022 ; Shirazibeheshti et al, 2018 ; Wacongne et al, 2011 ): While the MMN is primarily sensitive to local regularities, such as basic stimulus repetition, the P3 is additionally sensitive to global regularities such as a repeating pattern over an extended period of time. In addition to global sequence monitoring with respect to temporal regularities, the P3 appears to additionally be sensitive to global sequence regularities in terms of the multi‐modal stimulus configuration (Grundei et al, 2023 ).…”

“…Furthermore, it has been proposed that the deviance detection system of MMN and P3 differentially responds to expectation violation to sequence regularities on different levels of complexity. Studies employing the “local–global” paradigm (Bekinschtein et al, 2009 ) in which stimulus sequences are defined by local regularities (e.g., the tendency of a stimulus to repeat) and additional global regularities (e.g., every fifth stimulus in a repeated sequence is a deviant) show that the MMN is only elicited by the local regularity violations whereas the later P3 is additionally sensitive to violations of the global deviant regularity (Bekinschtein et al, 2009 ; Chennu et al, 2013 ; Chennu et al, 2016 ; Dürschmid et al, 2016 ; El Karoui et al, 2015 ; King et al, 2014 ; Niedernhuber et al, 2022 ; Shirazibeheshti et al, 2018 ; Wacongne et al, 2011 ). Strikingly, this dichotomy for MMRs was recently shown to hold for the auditory, somatosensory and visual modality alike (Niedernhuber et al, 2022 ).…”

“…Studies employing the “local–global” paradigm (Bekinschtein et al, 2009 ) in which stimulus sequences are defined by local regularities (e.g., the tendency of a stimulus to repeat) and additional global regularities (e.g., every fifth stimulus in a repeated sequence is a deviant) show that the MMN is only elicited by the local regularity violations whereas the later P3 is additionally sensitive to violations of the global deviant regularity (Bekinschtein et al, 2009 ; Chennu et al, 2013 ; Chennu et al, 2016 ; Dürschmid et al, 2016 ; El Karoui et al, 2015 ; King et al, 2014 ; Niedernhuber et al, 2022 ; Shirazibeheshti et al, 2018 ; Wacongne et al, 2011 ). Strikingly, this dichotomy for MMRs was recently shown to hold for the auditory, somatosensory and visual modality alike (Niedernhuber et al, 2022 ). Evidence converges to the view that the MMN, induced by local deviants primarily activates sensory regions, while the P3 MMR after global deviance is accompanied by frontal (Chao et al, 2018 ; Chennu et al, 2013 ; El Karoui et al, 2015 ) and fronto‐parietal activations (Bekinschtein et al, 2009 ; Uhrig et al, 2014 ), in line with the neuronal sources thought to underlie the P3 (Linden, 2005 ; Polich, 2007 ).…”

Why mismatch negativity continues to hold potential in probing altered brain function in schizophrenia

“…Moreover, different brain regions are purported to extract regularity information over different temporal intervals such that shorter timescale information that can be gleaned by sensory areas is modulated by longer timescale information supported by more rostral areas, such as the prefrontal cortex 136–138 . Patterns that are only evident over longer timescales can therefore theoretically be used to weight the dependence of MMN on higher areas of the cortical network 139–142 . Finally, theoretical perspectives on the varied reasons for different MMN amplitudes have also progressed.…”

“… 136 , 137 , 138 Patterns that are only evident over longer timescales can therefore theoretically be used to weight the dependence of MMN on higher areas of the cortical network. 139 , 140 , 141 , 142 Finally, theoretical perspectives on the varied reasons for different MMN amplitudes have also progressed. What was once considered probabilistic inference based on sensory memory 5 has been augmented by evidence of statistical learning of transitional probability 143 and even speculation that the system is engaged in hierarchical inference.…”