Decoding the visual and subjective contents of the human brain

Kamitani, Yukiyasu; Tong, Frank

doi:10.1038/nn1444

Cited by 1,678 publications

(1,657 citation statements)

References 38 publications

Supporting

Mentioning

1,556

Contrasting

Unclassified

Order By: Relevance

“…The main analyses were based on ROI decoding (Haynes and Rees, 2005;Kamitani and Tong, 2005), in which the voxels of a given ROI in native space were used for classification. ROI decoding followed the same cross-validation procedure as detailed for the searchlight analysis.…”

Section: Multivariate Analysesmentioning

confidence: 99%

Non-holistic coding of objects in lateral occipital complex with and without attention

et al. 2015

View full text Add to dashboard Cite

A fundamental issue in visual cognition is whether high-level visual areas code objects in a part-based or a view-based (holistic) format. By examining the viewpoint invariance of object recognition, previous behavioral and neuroimaging studies have yielded ambiguous results, supporting both types of representational formats. A critical factor distinguishing the two formats could be the availability of attentional resources, as a number of studies have found greater viewpoint invariance for attended compared to unattended objects. It has therefore been suggested that attention is necessary to enable part-based representations, whereas holistic representations are automatically activated irrespective of attention. In this functional magnetic resonance imaging study we used a multivariate approach to probe the format of object representations in human lateral occipital complex (LOC) and its dependence on attention. We presented human participants with intact and half-split versions of objects that were either attended or unattended. Cross-classifying between intact and split objects, we found that the objectrelated information coded in activation patterns of intact objects is fully preserved in the patterns of split objects and vice versa. Importantly, the generalization between intact and split objects did not depend on attention. Our findings demonstrate that LOC codes objects in a non-holistic format, both in the presence and absence of attention.

show abstract

Section: Multivariate Analysesmentioning

confidence: 99%

Non-holistic coding of objects in lateral occipital complex with and without attention

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Although columnar-scale imaging has been reported in animals 135 , the small scale of this organization precludes imaging of the individual columns in humans with current methods (see figure, part c, righthand panel; the colour scale represents preferred orientation with hue and strength of selectivity with colour saturation). Nevertheless, recent fMRI studies have been able to exploit subtle differences between voxels in their selectivity for oriented gratings to decode the orientation of the gratings from the distributed activation pattern in human V1, using multi-voxel pattern analyses 22,23,136 (see figure, part c, right-hand panel). In addition to this organization by object category, a weak eccentricity bias that appears to be an extension of the eccentricity map in retinotopic visual areas has been reported in extrastriate and temporal visual cortex 30,31 .…”

Section: Box 1 | Recent Advances Through Functional Mrimentioning

confidence: 99%

Interpreting fMRI data: maps, modules and dimensions

2008

View full text Add to dashboard Cite

Neuroimaging research over the past decade has revealed a detailed picture of the functional organization of the human brain. Here we focus on two fundamental questions that are raised by the detailed mapping of sensory and cognitive functions and illustrate these questions with findings from the object-vision pathway. First, are functionally specific regions that are located close together best understood as distinct cortical modules or as parts of a larger-scale cortical map? Second, what functional properties define each cortical map or module? We propose a model in which overlapping continuous maps of simple features give rise to discrete modules that are selective for complex stimuli.Advances in brain imaging technology (especially functional MRI (fMRI)) have radically improved our understanding of the functional organization of the human brain (BOX 1). In this Review we describe the organization of the ventral visual pathway, which is characterized by strong selectivity for particular object categories (for example, faces and bodies) at the level of both individual neurons and larger cortical regions. We then consider two central questions: whether this organization reflects maps or modules, and what properties are mapped. In each case we derive clues from the literature on the primary sensory cortex, in which cortical maps have been studied extensively using electrophysiology in animals. We find that apparently modular cortical regions, such as orientation columns and face-selective regions, might be parts of larger maps, and show that it is a substantial challenge to determine the basic properties and dimensions that describe functional organization most parsimoniously. We then propose a new framework that reconciles the existence of graded cortical maps and distinct functional modules. In this framework, the strong category selectivity that exists for faces and other objects might arise from the nonlinear combination of multiple correlated maps for simpler stimulus properties. The ventral visual pathwayThe ventral visual pathway comprises a large cortical region that occupies the ventral and lateral surfaces of the occipital and temporal lobes (FIG. 1). A substantial proportion of fMRI voxels in this pathway are 'object-selective' -that is, they respond more strongly when people view images of objects than when people view scrambled versions of these objects or texture patterns. This object-selective region is often referred to as the lateral occipital complex (LOC) 1 . The LOC has little selectivity for particular stimulus categories 2-4 , but several regions of NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript cortex near the LOC are selective for particular object categories: they respond at least twice as strongly to their 'preferred' stimuli than to other stimuli. For example, in essentially all humans cortical regions can be found that respond selectively to faces (the fusiform face area (FFA) 5,6 and, in many individuals, the occipital face area (OFA)) 7,8 , to pl...

show abstract

“…Visual gratings were chosen as stimuli because their encoding in visual cortical response patterns is reasonably well understood. In particular, we examined the accuracy of multivariate decoding in the primary visual cortex (V1), which is generally high (Alink, Krugliak, Walther, & Kriegeskorte, 2013; Kamitani & Tong, 2005; Tong et al, 2010). …”

Section: Introductionmentioning

confidence: 99%

Prospective motion correction improves the sensitivity of fMRI pattern decoding

Huang

Carlin

Alink

et al. 2018

Human Brain Mapping

View full text Add to dashboard Cite

We evaluated the effectiveness of prospective motion correction (PMC) on a simple visual task when no deliberate subject motion was present. The PMC system utilizes an in‐bore optical camera to track an external marker attached to the participant via a custom‐molded mouthpiece. The study was conducted at two resolutions (1.5 mm vs 3 mm) and under three conditions (PMC On and Mouthpiece On vs PMC Off and Mouthpiece On vs PMC Off and Mouthpiece Off). Multiple data analysis methods were conducted, including univariate and multivariate approaches, and we demonstrated that the benefit of PMC is most apparent for multi‐voxel pattern decoding at higher resolutions. Additional testing on two participants showed that our inexpensive, commercially available mouthpiece solution produced comparable results to a dentist‐molded mouthpiece. Our results showed that PMC is increasingly important at higher resolutions for analyses that require accurate voxel registration across time.

show abstract

Decoding the visual and subjective contents of the human brain

Cited by 1,678 publications

References 38 publications

Non-holistic coding of objects in lateral occipital complex with and without attention

Non-holistic coding of objects in lateral occipital complex with and without attention

Interpreting fMRI data: maps, modules and dimensions

Prospective motion correction improves the sensitivity of fMRI pattern decoding

Contact Info

Product

Resources

About