A neural code for three-dimensional object shape in macaque inferotemporal cortex

Yamane, Yoshihiro; Carlson, Eric T.; Bowman, Katherine C; Wang, Zhihong; Connor, Charles E.

doi:10.1038/nn.2202

Cited by 262 publications

(276 citation statements)

References 47 publications

Supporting

Mentioning

247

Contrasting

Order By: Relevance

“…We explored neuronal responses to textures along this space, although the stimuli themselves were synthesized using all 740 parameters. To efficiently collect preferred textures for individual neurons, we adopted an adaptive sampling procedure (25) (Fig. 1B).…”

Section: Resultsmentioning

confidence: 99%

“…3). The fundamental difficulty of data sampling in the great diversity of natural textures was overcome by extending an adaptive sampling procedure (25) to explore the manifold of natural textures in a high-dimensional sampling space. We successfully modeled the obtained responses of individual cells as a linear tuning to sparse combinations of image statistics.…”

Section: Discussionmentioning

confidence: 99%

“…We introduced the adaptive sampling procedure to efficiently search parameter space for finding effective textural stimuli (25). In this procedure, we first randomly selected 50 textures (the first generation) from the 10,355-image set and recorded the single-cell responses they elicited.…”

Section: Methodsmentioning

confidence: 99%

“…Indeed, previous studies on neural texture representation were typically based on strictly limited samples (6-8)-e.g., 15 different textures were used in Freeman et al (7)-despite a potentially large number of parameters describing the neural selectivity for various natural textures. In this study, we overcome this problem of dimensionality by combining the parametric texture synthesis with an adaptive sampling procedure (23)(24)(25) that efficiently sampled a portion of the stimuli that were expected to evoke stronger responses in the recorded neuron, as well as linear regression with a dimension-reduced version of the PS statistics. As a result, we successfully fitted neural responses to hundreds of textures using PS statistics.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Image statistics underlying natural texture selectivity of neurons in macaque V4

Okazawa

Tajima

Komatsu

2014

Proc. Natl. Acad. Sci. U.S.A.

146

171

View full text Add to dashboard Cite

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.texture perception | material perception | visual area V4 | single-cell recording | image analysis I n the visual world, objects are characterized in part by their shapes, but also by their textures (1). The wide variety of textures we experience enables us to segment objects from backgrounds, perceive object properties, and recognize materials. It is well established that the representation of complex shapes and contours is gradually built up along the ventral visual pathway (2-5). On the other hand, how textural information is processed in the cortex is largely unknown, although some recent studies have examined the representation of natural textures and surface properties in the macaque ventral visual areas (6-11). Because, unlike contours, textures cannot be described based on combinations of edge fragments, we need to consider different underlying cortical processing.In contrast to the limited knowledge available from physiology, computational descriptions of textures have been extensively developed in the fields of psychophysics (12-16) and computer vision (17)(18)(19). In one such description, Portilla and Simoncelli (20) proposed that textures could be represented using an ensemble of summary statistics, including features derived from the luminance histogram and the amplitudes of the outputs of Gabor-like filters, as well as higher-order statistics such as the correlations across the filter outputs (see Fig. 3 for details; hereafter, we call this collection of statistics "PS statistics," using the authors' initials). Portilla and Simoncelli successfully generated new textures that were visually indistinguishable from the originals solely by making their PS statistics identical. Their algorithm is particularly inspiring because PS statistics use filters and computations that share biological properties. It was recently shown, for example, that a version of their synthesis algorithm can generate perceptually indistinguishable visual images (visual metamers) (21) and that naturalisti...

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Image statistics underlying natural texture selectivity of neurons in macaque V4

Okazawa

Tajima

Komatsu

2014

Proc. Natl. Acad. Sci. U.S.A.

146

171

View full text Add to dashboard Cite

show abstract

“…A crucial stage in any theory of 3D vision must include an explanation of how the visual system extracts the key information from the image. At present, there is an explanatory gap between the known response properties of cells early in the visual processing hierarchy, which measure local 2D image features (10)(11)(12)(13)(14)(15)(16), and cells higher in the processing stream, which respond to various 3D shape properties (17)(18)(19)(20)(21)(22)(23)(24). How does the brain put the measurements made in the primary visual cortex (V1) to good use to arrive at an estimate of 3D shape?…”

mentioning

confidence: 99%

Estimation of 3D shape from image orientations

Fleming

Holtmann-Rice

Bülthoff

2011

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

One of the main functions of vision is to estimate the 3D shape of objects in our environment. Many different visual cues, such as stereopsis, motion parallax, and shading, are thought to be involved. One important cue that remains poorly understood comes from surface texture markings. When a textured surface is slanted in 3D relative to the observer, the surface patterns appear compressed in the retinal image, providing potentially important information about 3D shape. What is not known, however, is how the brain actually measures this information from the retinal image. Here, we explain how the key information could be extracted by populations of cells tuned to different orientations and spatial frequencies, like those found in the primary visual cortex. To test this theory, we created stimuli that selectively stimulate such cell populations, by "smearing" (filtering) images of 2D random noise into specific oriented patterns. We find that the resulting patterns appear vividly 3D, and that increasing the strength of the orientation signals progressively increases the sense of 3D shape, even though the filtering we apply is physically inconsistent with what would occur with a real object. This finding suggests we have isolated key mechanisms used by the brain to estimate shape from texture. Crucially, we also find that adapting the visual system's orientation detectors to orthogonal patterns causes unoriented random noise to look like a specific 3D shape. Together these findings demonstrate a crucial role of orientation detectors in the perception of 3D shape.shape perception | surface perception | orientation field | complex cells W hen we look at a textured object, the projection of the surface markings into the retinal image compresses them in ways that can indicate the object's 3D shape. This compression has two distinct causes. The first cause is distance-dependent: when a surface patch is moved further away from the eye, the texture shrinks isotropically in the image as a function of the distance. The second cause of compression is foreshortening: when a surface is slanted relative to the line of sight, the texture is anisotropically compressed along the direction of the slant, with greater slant leading to greater compression.It is well known that the visual system can use these texture compression cues to estimate 3D shape (1-9). What is not known, however, is how the visual system measures the compression at each point in the image. A crucial stage in any theory of 3D vision must include an explanation of how the visual system extracts the key information from the image. At present, there is an explanatory gap between the known response properties of cells early in the visual processing hierarchy, which measure local 2D image features (10-16), and cells higher in the processing stream, which respond to various 3D shape properties (17-24). How does the brain put the measurements made in the primary visual cortex (V1) to good use to arrive at an estimate of 3D shape?Estimating the extent and direction of ...

show abstract

Visual Processing in the Primate Brain

Baker

2012

Handbook of Psychology, Second Edition

View full text Add to dashboard Cite

A neural code for three-dimensional object shape in macaque inferotemporal cortex

Cited by 262 publications

References 47 publications

Image statistics underlying natural texture selectivity of neurons in macaque V4

Image statistics underlying natural texture selectivity of neurons in macaque V4

Estimation of 3D shape from image orientations

Visual Processing in the Primate Brain

Contact Info

Product

Resources

About