Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

Kumar, Sreejan; Ellis, Cameron T.; O'Connell, Thomas; Chun, Marvin M.; Turk‐Browne, Nicholas B.

doi:10.1371/journal.pcbi.1008457

Cited by 11 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, regions in neocortical grey matter commonly thought to be orthogonal to the task in univariate analyses in fact contain robust task-related information. These findings complement recent work using statistical learning to optimize voxel weights for predictive performance [8][9][10][11][12] and demonstrate that the presence of information is far more distributed across the brain than previously thought. Moreover, our analyses establish for the first time how accessible this information truly is: our models simply use mass-univariate t-statistics without any regularization or consideration of the t-statistics' joint distribution.…”

supporting

confidence: 83%

“…Human fMRI work demonstrates that 100 repetitions of the same task (three participants, 9-10 sessions over three months) can uncover neocortex-wide information 19 . Decoding studies, which rely on statistical learning approaches, evidence the existence of taskspecific information outside of GLM areas [9][10][11][12][13] ; it has also been demonstrated that some of these local information patterns can be uncovered via multivariate decoding methods [20][21][22] . Similarly, recent human and macaque monkey fMRI studies demonstrate the presence of retinotopic tuning in cortical and, in macaques, subcortical regions remote from the visual cortex 23,24 .…”

Section: Figure 4 Task-relevant Information Is Pervasively Present Th...mentioning

confidence: 99%

See 1 more Smart Citation

Widespread, perception-related information in the human brain scales with levels of consciousness

Vigotsky

Jabakhanji

Branco

et al. 2022

Preprint

View full text Add to dashboard Cite

How does the human brain generate coherent, subjective perceptions—transforming yellow and oblong visual sensory information into the perception of an edible banana? This is a hard problem. The standard viewpoint posits that anatomical and functional networks somehow integrate local, specialized processing across the brain to construct unique percepts. Here, we provide evidence for a novel organizational concept: we uncover task-specific information distributed across the human brain. We use functional magnetic resonance imaging (fMRI)-based decoding models to probe the presence of task-specific information. In doing so, we show that brain regions that are empirically almost entirely task-independent (considered to be "noise"—i.e., t-statistics ≈ 0—in standard univariate fMRI analyses) can decode various tasks, even without statistical learning tools. Thus, task-specific information is present and readily accessible throughout the neocortex, challenging the sensitivity of traditional linear, univariate analytical approaches. Remarkably, even subcortical structures and the cerebellum contain task-specific signals in voxels that would univariately be considered "noise". Finally, using data from an auditory study with different levels of sedation, we show that the widespread signal in regions remote from the primary and secondary sensory cortices (auditory cortex) depends on the level of sedation, suggesting the brain's widespread, task-specific information is related to perception rather than the encoding of the sensory stimulus. Our analysis uncovers task-specific and consciousness level-dependent information across the human brain, which we hypothesize to be the basis for coherent, subjective perceptions.

show abstract

supporting

confidence: 83%

Section: Figure 4 Task-relevant Information Is Pervasively Present Th...mentioning

confidence: 99%

Widespread, perception-related information in the human brain scales with levels of consciousness

Vigotsky

Jabakhanji

Branco

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…By focusing on speech representations, the present study provides four empirical contributions to the investigation of auditory representations in brains and deep learning models (Yamins & DiCarlo, 2016;Keshishian et al, 2020;Huang et al, 2019;Liu et al, 2021;Kell et al, 2018;Kumar et al, 2020;Koumura et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

“…speech-to-text using Dutch, English or Bengali). Previous studies have already shown that deep convolutional neural networks trained to classify words, musical genres (Kell et al, 2018;Kumar et al, 2020) or natural sounds (Koumura et al, 2019), generate brain-like representations, in the sense that one can find a linear correspondence between the activation of the neural networks and the activations of the brain (Figure 1). This similarity can be quantified with a "brain score" (Jung et al, 2019), a correlation between the brain measurements and a linear projection of the model's activations, under the assumption that representations are defined as linearly exploitable information (Hung et al, 2005;Kamitani & Tong, 2005;Kriegeskorte & Kievit, 2013;King et al, 2018).…”

Section: Introductionmentioning

confidence: 98%

Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech

Millet¹,

J²

2021

Preprint

View full text Add to dashboard Cite

Our ability to comprehend speech remains, to date, unrivaled by deep learning models. This feat could result from the brain’s ability to fine-tune generic sound representations for speech-specific processes. To test this hypothesis, we compare i) five types of deep neural networks to ii) human brain responses elicited by spoken sentences and recorded in 102 Dutch subjects using functional Magnetic Resonance Imaging (fMRI). Each network was either trained on an acoustics scene classification, a speech-to-text task (based on Bengali, English, or Dutch), or not trained. The similarity between each model and the brain is assessed by correlating their respective activations after an optimal linear projection. The differences in brain-similarity across networks revealed three main results. First, speech representations in the brain can be accounted for by random deep networks. Second, learning to classify acoustic scenes leads deep nets to increase their brain similarity. Third, learning to process phonetically-related speech inputs (i.e., Dutch vs English) leads deep nets to reach higher levels of brain-similarity than learning to process phonetically-distant speech inputs (i.e. Dutch vs Bengali). Together, these results suggest that the human brain fine-tunes its heavily-trained auditory hierarchy to learn to process speech.

show abstract

“…We previously validated that this fitting procedure produces accurate simulations of real data [ 23 ]. We have used fmrisim to evaluate the power of different experimental design parameters [ 23 ] and also to evaluate the efficacy of new analysis methods [ 86 , 87 ].…”

Section: Methods In Brainiakmentioning

confidence: 99%

BrainIAK: The Brain Imaging Analysis Kit

et al. 2022

Self Cite

View full text Add to dashboard Cite

Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEMs), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high-performance compute (HPC) clusters, and the same code can be seamlessly transferred from a laptop to a cluster. For each of the aforementioned techniques, we describe the data analysis problem that the technique is meant to solve and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.

show abstract

Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

Cited by 11 publications

References 25 publications

Widespread, perception-related information in the human brain scales with levels of consciousness

Widespread, perception-related information in the human brain scales with levels of consciousness

Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech

BrainIAK: The Brain Imaging Analysis Kit

Contact Info

Product

Resources

About