Image statistics underlying natural texture selectivity of neurons in macaque V4

Abstract

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.texture perception | material perception | visual area V4 | single-cell recording | image analysis I n the visual world, objects are characterized in part by their shapes, but also by their textures (1). The wide variety of textures we experience enables us to segment objects from backgrounds, perceive object properties, and recognize materials. It is well established that the representation of complex shapes and contours is gradually built up along the ventral visual pathway (2-5). On the other hand, how textural information is processed in the cortex is largely unknown, although some recent studies have examined the representation of natural textures and surface properties in the macaque ventral visual areas (6-11). Because, unlike contours, textures cannot be described based on combinations of edge fragments, we need to consider different underlying cortical processing.In contrast to the limited knowledge available from physiology, computational descriptions of textures have been extensively developed in the fields of psychophysics (12-16) and computer vision (17)(18)(19). In one such description, Portilla and Simoncelli (20) proposed that textures could be represented using an ensemble of summary statistics, including features derived from the luminance histogram and the amplitudes of the outputs of Gabor-like filters, as well as higher-order statistics such as the correlations across the filter outputs (see Fig. 3 for details; hereafter, we call this collection of statistics "PS statistics," using the authors' initials). Portilla and Simoncelli successfully generated new textures that were visually indistinguishable from the originals solely by making their PS statistics identical. Their algorithm is particularly inspiring because PS statistics use filters and computations that share biological properties. It was recently shown, for example, that a version of their synthesis algorithm can generate perceptually indistinguishable visual images (visual metamers) (21) and that naturalisti...

Functional implications of orientation maps in primary visual cortex

“…The importance of cells that extract contours and edges has been recognized since Hubel and Wiesel1; however, extra-striate cells that are selective for specific classes of patterns and textures have only been reported recently2125. Our empirical results and model simulations suggest that cells in pinwheel centres provide suitable inputs for pattern selective cells, whereas cells in iso-orientation domains enhance sensitivity to edges.…”

“…Cortical processing of isolated orientations is important for extracting contours and edges1819, three-dimensional shape from shading20 and the direction of local motion1. However, processing multiple orientations simultaneously is essential for more sophisticated visual analyses, such as extracting image patterns21, pattern motion2223, pattern symmetry24, material properties25 and three-dimensional shape from texture262728. Even though processing stimuli with multiple orientations is important, the cortical responses to those stimuli are thought to be strongly suppressed by a neuronal mechanism known as cross-orientation suppression.…”

Selectivity and tolerance for visual texture in macaque V2

“…Thus, most neurons saw only a portion of the stimuli, over which the statistics would not have been fully matched. Finally, recall that the transformation from V1 to V2 is part of a cascade, and it may well be that perception relies more on downstream areas, such as V4, where neurons may show even more selectivity and tolerance for the statistics we used (31,45).…”

Representation of Naturalistic Image Structure in the Primate Visual Cortex

“…The same is true for V2—knowing that the population of V2 neurons can represent statistical structure in images does not uniquely specify the response properties of individual V2 neurons, and a major challenge for ongoing research will be to create such models. Related work in V4 has explored computations that produce some forms of texture specificity (Okazawa et al 2015), and we have begun to explore models to account for the texture responses of V2 neurons. The structure of the statistical model from which the texture stimuli are generated (Portilla and Simoncelli 2000) provides a starting point.…”