Underlying principles of visual shape selectivity in posterior inferotemporal cortex

Brincat, Scott L.; Connor, Charles E.

doi:10.1038/nn1278

Cited by 357 publications

(345 citation statements)

References 48 publications

Supporting

Mentioning

308

Contrasting

Order By: Relevance

“…Network depth ranged from one to three layers, and filter weights for each layer were chosen randomly from bounded uniform distributions whose bounds were model parameters (SI Text). These models are consistent with the Hierarchical Linear-Nonlinear (HLN) hypothesis that higher level neurons (e.g., IT) output a linear weighting of inputs from intermediate-level (e.g., V4) neurons followed by simple additional nonlinearities (14,16,29).…”

Section: Resultssupporting

confidence: 80%

Performance-optimized hierarchical models predict neural responses in higher visual cortex

Yamins

Hong

Cadieu

et al. 2014

Proc. Natl. Acad. Sci. U.S.A.

1,653

1,797

View full text Add to dashboard Cite

The ventral visual stream underlies key human visual object recognition abilities. However, neural encoding in the higher areas of the ventral stream remains poorly understood. Here, we describe a modeling approach that yields a quantitatively accurate model of inferior temporal (IT) cortex, the highest ventral cortical area. Using high-throughput computational techniques, we discovered that, within a class of biologically plausible hierarchical neural network models, there is a strong correlation between a model's categorization performance and its ability to predict individual IT neural unit response data. To pursue this idea, we then identified a high-performing neural network that matches human performance on a range of recognition tasks. Critically, even though we did not constrain this model to match neural data, its top output layer turns out to be highly predictive of IT spiking responses to complex naturalistic images at both the single site and population levels. Moreover, the model's intermediate layers are highly predictive of neural responses in the V4 cortex, a midlevel visual area that provides the dominant cortical input to IT. These results show that performance optimization-applied in a biologically appropriate model classcan be used to build quantitative predictive models of neural processing.computational neuroscience | computer vision | array electrophysiology R etinal images of real-world objects vary drastically due to changes in object pose, size, position, lighting, nonrigid deformation, occlusion, and many other sources of noise and variation. Humans effortlessly recognize objects rapidly and accurately despite this enormous variation, an impressive computational feat (1). This ability is supported by a set of interconnected brain areas collectively called the ventral visual stream (2, 3), with homologous areas in nonhuman primates (4). The ventral stream is thought to function as a series of hierarchical processing stages (5-7) that encode image content (e.g., object identity and category) increasingly explicitly in successive cortical areas (1,8,9). For example, neurons in the lowest area, V1, are well described by Gabor-like edge detectors that extract rough object outlines (10), although the V1 population does not show robust tolerance to complex image transformations (9). Conversely, rapidly evoked population activity in top-level inferior temporal (IT) cortex can directly support realtime, invariant object categorization over a wide range of tasks (11,12). Midlevel ventral areas-such as V4, the dominant cortical input to IT-exhibit intermediate levels of object selectivity and variation tolerance (12-14).Significant progress has been made in understanding lower ventral areas such as V1, where conceptually compelling models have been discovered (10). These models are also quantitatively accurate and can predict response magnitudes of individual neuronal units to novel image stimuli. Higher ventral cortical areas, especially V4 and IT, have been much more difficult to understand. Al...

show abstract

Section: Resultssupporting

confidence: 80%

Performance-optimized hierarchical models predict neural responses in higher visual cortex

Yamins

Hong

Cadieu

et al. 2014

Proc. Natl. Acad. Sci. U.S.A.

1,653

1,797

View full text Add to dashboard Cite

show abstract

“…A different study (25) recorded neuronal responses to complex shapes in TEO and/or posterior TE, where we found MCP to be located. These stimuli were 2D line drawings composed of straight lines and curved components.…”

Section: Discussionmentioning

confidence: 67%

Curvature-processing network in macaque visual cortex

Yue

Pourladian

Tootell

et al. 2014

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Our visual environment abounds with curved features. Thus, the goal of understanding visual processing should include the processing of curved features. Using functional magnetic resonance imaging in behaving monkeys, we demonstrated a network of cortical areas selective for the processing of curved features. This network includes three distinct hierarchically organized regions within the ventral visual pathway: a posterior curvature-biased patch (PCP) located in the near-foveal representation of dorsal V4, a middle curvature-biased patch (MCP) located on the ventral lip of the posterior superior temporal sulcus (STS) in area TEO, and an anterior curvature-biased patch (ACP) located just below the STS in anterior area TE. Our results further indicate that the processing of curvature becomes increasingly complex from PCP to ACP. The proximity of the curvature-processing network to the well-known face-processing network suggests a possible functional link between them.curvature patches | face patches | curved Gabor filters D ecades of research have focused on understanding visual feature processing, particularly along the ventral visual pathway. Such studies have shown that neurons in lower-order visual areas (e.g., V1) respond strongly to simple oriented contours (1), whereas neurons in higher-order visual areas (e.g., inferior temporal cortex) respond selectively to more complex image features and/or visual categories (2-4), in ways that are not yet fully understood. To link these extremes in visual information processing, many studies have aimed to clarify the optimal "trigger" features at intermediate levels of the visual cortical hierarchy.Among these features, stimulus curvature has not been well studied. This is surprising because, strictly, all lines are curved to some extent, except for the single exception of a perfectly straight line. This ubiquity of curved shapes also extends to 3D surfaces (5). In nature, where much of our visual system presumably evolved, perfectly flat surfaces are rare. Even the flattest of natural features (e.g., oceans, sandy beaches) are often curved to some extent, due to wind, water motion, and even the curvature of the earth. Thus, it is important to understand curvature processing to fully unravel the steps in cortical visual processing.Among the few studies to test single neuron responses to curvature per se, Gallant et al. (6,7) reported that a significant percentage of neurons in macaque cortical area V4 is selective for curved stimuli. Intriguingly, these authors also noted that neurons preferring curved patterns were often anatomically clustered together. Subsequently, demonstrated that neurons in the parafoveal representation of dorsal V4 respond robustly to the curvature component of complex shapes. To our knowledge, there have been no systematic studies of curvature at levels below V4 in macaques.Intriguingly, some evidence suggests that the processing of curvature may interact selectively with the processing of faces. For instance, perceptual deficits in face reco...

show abstract

“…4). Furthermore, it has been shown that single cells and columns in the monkey IT cortex are selective for moderately complex features 96,97 ; that neurons in v4 and in the posterior IT cortex are tuned for simple shape characteristics 98,99 ; and that IT neurons exhibit gradual tuning in simple shape spaces 50,51 . Finally, single-cell recordings and fMRI studies have provided evidence of selectivity for object parts and non-accidental properties (for example, symmetry, parallelism and collinearity) in the ventral temporal cortex 49,100 , as predicted by structural description theories of object recognition 101,102 .…”

Section: Basic Properties In the Ventral Visual Cortexmentioning

confidence: 99%

Interpreting fMRI data: maps, modules and dimensions

2008

View full text Add to dashboard Cite

Neuroimaging research over the past decade has revealed a detailed picture of the functional organization of the human brain. Here we focus on two fundamental questions that are raised by the detailed mapping of sensory and cognitive functions and illustrate these questions with findings from the object-vision pathway. First, are functionally specific regions that are located close together best understood as distinct cortical modules or as parts of a larger-scale cortical map? Second, what functional properties define each cortical map or module? We propose a model in which overlapping continuous maps of simple features give rise to discrete modules that are selective for complex stimuli.Advances in brain imaging technology (especially functional MRI (fMRI)) have radically improved our understanding of the functional organization of the human brain (BOX 1). In this Review we describe the organization of the ventral visual pathway, which is characterized by strong selectivity for particular object categories (for example, faces and bodies) at the level of both individual neurons and larger cortical regions. We then consider two central questions: whether this organization reflects maps or modules, and what properties are mapped. In each case we derive clues from the literature on the primary sensory cortex, in which cortical maps have been studied extensively using electrophysiology in animals. We find that apparently modular cortical regions, such as orientation columns and face-selective regions, might be parts of larger maps, and show that it is a substantial challenge to determine the basic properties and dimensions that describe functional organization most parsimoniously. We then propose a new framework that reconciles the existence of graded cortical maps and distinct functional modules. In this framework, the strong category selectivity that exists for faces and other objects might arise from the nonlinear combination of multiple correlated maps for simpler stimulus properties. The ventral visual pathwayThe ventral visual pathway comprises a large cortical region that occupies the ventral and lateral surfaces of the occipital and temporal lobes (FIG. 1). A substantial proportion of fMRI voxels in this pathway are 'object-selective' -that is, they respond more strongly when people view images of objects than when people view scrambled versions of these objects or texture patterns. This object-selective region is often referred to as the lateral occipital complex (LOC) 1 . The LOC has little selectivity for particular stimulus categories 2-4 , but several regions of NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript cortex near the LOC are selective for particular object categories: they respond at least twice as strongly to their 'preferred' stimuli than to other stimuli. For example, in essentially all humans cortical regions can be found that respond selectively to faces (the fusiform face area (FFA) 5,6 and, in many individuals, the occipital face area (OFA)) 7,8 , to pl...

show abstract

Underlying principles of visual shape selectivity in posterior inferotemporal cortex

Cited by 357 publications

References 48 publications

Performance-optimized hierarchical models predict neural responses in higher visual cortex

Performance-optimized hierarchical models predict neural responses in higher visual cortex

Curvature-processing network in macaque visual cortex

Interpreting fMRI data: maps, modules and dimensions

Contact Info

Product

Resources

About