Representation of Naturalistic Image Structure in the Primate Visual Cortex

Movshon, J. Anthony; Simoncelli, Eero P.

doi:10.1101/sqb.2014.79.024844

Cited by 40 publications

(45 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work in macaque (Rust & Dicarlo, 2010) has found that responses in IT differ between intact and scrambled versions of the same image to a greater degree than do the responses in V4 when the Portilla-Simoncelli algorithm is used. A recent report using fMRI in humans (Freeman, Ziemba, Simoncelli, & Movshon, 2013; Movshon & Simoncelli, 2014) has contrasted responses to Portilla-Simoncelli scrambled textures and intact natural textures and found differential responses occurred only at and beyond area V4. Our approach may thus make the resulting SSVEP more selective to the intrinsic structure of orthography and faces than other approaches such as phase scrambling.…”

Section: Discussionmentioning

confidence: 99%

Temporal Tuning of Word- and Face-selective Cortex

Yeatman

Norcia

2016

Journal of Cognitive Neuroscience

View full text Add to dashboard Cite

Sensitivity to temporal change places fundamental limits on object processing in the visual system. An emerging consensus from the behavioral and neuroimaging literature suggests that temporal resolution differs substantially for stimuli of different complexity and for brain areas at different levels of the cortical hierarchy. Here we used Steady-State Visual Evoked Potentials (SSVEPs) to directly measure three fundamental parameters that characterize the underlying neural response to text and face images: temporal resolution, peak temporal frequency and response latency. We presented full-screen images of text or a human face, alternated with a scrambled image, at temporal frequencies between 1 and 12 Hz. These images elicited a robust response at the first harmonic that showed differential tuning, scalp topography and delay for the text and face images. Face selective responses were maximal at 4 Hz, but text selective responses, by contrast were maximal at 1 Hz. The topography of the text image response was strongly left-lateralized at higher stimulation rates, while the response to the face image was slightly right-lateralized but nearly bilateral at all frequencies. Both text and face images elicited steady state activity at more than one apparent latency; we observed early (141ms–160ms) and late (>250ms) text and face selective responses. These differences in temporal tuning profiles are likely to reflect differences in the nature of the computations performed by word and face selective cortex. Despite the close proximity of word and face selective regions on the cortical surface, our measurements demonstrate substantial differences in the temporal dynamics of word-versus face-selective responses.

show abstract

Section: Discussionmentioning

confidence: 99%

Temporal Tuning of Word- and Face-selective Cortex

Yeatman

Norcia

2016

Journal of Cognitive Neuroscience

View full text Add to dashboard Cite

show abstract

“…The parametric texture model of Portilla and Simoncelli (Portilla & Simoncelli, 2000;Simoncelli & Portilla, 1998) extended this approach by additionally matching the correlations between channels and other statistics, producing more realistic appearance matches to textures. This model has since had broad impact on the field of human perception and neuroscience: the texture statistic representation may provide a fruitful way to understand the processing in midventral visual areas (Freeman & Simoncelli, 2011;Freeman, Ziemba, Heeger, Simoncelli, & Movshon, 2013;Movshon & Simoncelli, 2014;Okazawa, Tajima, & Komatsu, 2015;Ziemba, Freeman, Movshon, & Simoncelli, 2016), and it has been argued to provide a good approximation of the type of information encoded in the periphery, and thus a model for tasks such as crowding and visual search (Balas, Nakano, & Rosenholtz, 2009;Freeman & Simoncelli, 2011;Keshvari & Rosenholtz, 2016;Rosenholtz, 2011;Rosenholtz, Huang, & Ehinger, 2012;Rosenholtz, Huang, Raj, Balas, & Ilie, 2012)-though other evidence questions the more general adequacy of this representation for explaining crowding and peripheral appearance (Agaoglu & Chung, 2016;Clarke, Herzog, & Francis, 2014;Herzog, Sayim, Chicherov, & Manassi, 2015;Wallis, Bethge, & Wichmann, 2016).…”

Section: Studying Texture Perception With Parametric Texture Modelsmentioning

confidence: 99%

A parametric texture model based on deep convolutional features closely matches texture appearance for humans

et al. 2017

View full text Add to dashboard Cite

Our visual environment is full of texture-"stuff" like cloth, bark, or gravel as distinct from "things" like dresses, trees, or paths-and humans are adept at perceiving subtle variations in material properties. To investigate image features important for texture perception, we psychophysically compare a recent parametric model of texture appearance (convolutional neural network [CNN] model) that uses the features encoded by a deep CNN (VGG-19) with two other models: the venerable Portilla and Simoncelli model and an extension of the CNN model in which the power spectrum is additionally matched. Observers discriminated model-generated textures from original natural textures in a spatial three-alternative oddity paradigm under two viewing conditions: when test patches were briefly presented to the near-periphery ("parafoveal") and when observers were able to make eye movements to all three patches ("inspection"). Under parafoveal viewing, observers were unable to discriminate 10 of 12 original images from CNN model images, and remarkably, the simpler Portilla and Simoncelli model performed slightly better than the CNN model (11 textures). Under foveal inspection, matching CNN features captured appearance substantially better than the Portilla and Simoncelli model (nine compared to four textures), and including the power spectrum improved appearance matching for two of the three remaining textures. None of the models we test here could produce indiscriminable images for one of the 12 textures under the inspection condition. While deep CNN (VGG-19) features can often be used to synthesize textures that humans cannot discriminate from natural textures, there is currently no uniformly best model for all textures and viewing conditions.

show abstract

“…The parametric texture model of Portilla and Simoncelli (Portilla & Simoncelli, 2000;Simoncelli & Portilla, 1998) extended this approach by additionally matching the correlations between channels and other statistics, producing more realistic appearance matches to textures. This model has since had broad impact on the field of human perception and neuroscience: the texture statistic representation may provide a fruitful way to understand the processing in mid-ventral visual areas (Freeman & Simoncelli, 2011;Freeman, Ziemba, Heeger, Simon-celli, & Movshon, 2013;Movshon & Simoncelli, 2014;Okazawa, Tajima, & Komatsu, 2015;Ziemba, Freeman, Movshon, & Simoncelli, 2016), and it has been argued to provide a good approximation of the type of information encoded in the periphery, and thus a model for tasks such as crowding and visual search (Balas, Nakano, & Rosenholtz, 2009;Freeman & Simoncelli, 2011;Keshvari & Rosenholtz, 2016;Rosenholtz, 2011;Rosenholtz, Huang, & Ehinger, 2012;Rosenholtz, Huang, Raj, Balas, & Ilie, 2012)-though other evidence questions the more general adequacy of this representation for explaining crowding and peripheral appearance (Agaoglu & Chung, 2016;Clarke, Herzog, & Francis, 2014;Herzog, Sayim, Chicherov, & Manassi, 2015;Wallis, Bethge, & Wichmann, 2016).…”

Section: Studying Texture Perception With Parametric Texture Modelsmentioning

confidence: 99%

A parametric texture model based on deep convolutional features closely matches texture appearance for humans

Wallis

Funke

Ecker

et al. 2017

Preprint

View full text Add to dashboard Cite

Our visual environment is full of texture-"stuff" like cloth, bark or gravel as distinct from "things" like dresses, trees or paths-and humans are adept at perceiving subtle variations in material properties. To investigate image features important for texture perception, we psychophysically compare a recent parameteric model of texture appearance (CNN model) that uses the features encoded by a deep convolutional neural network with two other models: the venerable Portilla and Simoncelli model (PS) and an extension of the CNN model in which the power spectrum is additionally matched. Observers discriminated model-generated textures from original natural textures in a spatial three-alternative oddity paradigm under two viewing conditions: when test patches were briefly presented to the near-periphery ("parafoveal") and when observers were able to make eye movements to all three patches ("inspection"). Under parafoveal viewing, observers were unable to discriminate 10 of 12 original images from CNN model images, and remarkably, the simpler PS model performed slightly better than the CNN model (11 textures). Under foveal inspection, matching CNN features captured appearance substantially better than the PS model (9 compared to 4 textures), and including the power spectrum improved appearance matching for two of the three remaining textures. None of the models we test here could produce indiscriminable images for one of the 12 textures under the inspection condition. While deep CNN (VGG-19) features can often be used to synthesise textures that humans cannot discriminate from natural textures, there is currently no uniformly best model for all textures and viewing conditions. Keywords: spatial vision, natural scenes, texture perception, peripheral vision . CC-BY 4.0 International license not peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was . http://dx.doi.org/10.1101/165761 doi: bioRxiv preprint first posted online IntroductionTextures are characterised by the repetition of smaller elements, sometimes with variation, to make up a pattern. Significant portions of the visual environment can be thought of as textures ("stuff" as distinct from "things"; Adelson & Bergen, 1991): your neighbour's pink floral wallpaper, the internal structure of dark German bread, the weave of a wicker basket, the gnarled bark of an old tree trunk, a bowl full of prawns ready for the barbie. Texture is an important material property whose perception is of adaptive value (Adelson, 2001;Fleming, 2014). For example, we can readily discriminate wet from dry stones (e.g. Ho, Landy, & Maloney, 2008), separating the underlying spatial texture from potentially temporary characteristics like glossiness. Where surfaces of different textures form occlusion boundaries, texture can provide a powerful segmentation cue; conversely, occlusion borders of similarly-textured surfaces can camoflage the occlusion (hiding a tiger among the leaves). Given the importance and ubiquity of visu...

show abstract

Representation of Naturalistic Image Structure in the Primate Visual Cortex

Cited by 40 publications

References 49 publications

Temporal Tuning of Word- and Face-selective Cortex

Temporal Tuning of Word- and Face-selective Cortex

A parametric texture model based on deep convolutional features closely matches texture appearance for humans

A parametric texture model based on deep convolutional features closely matches texture appearance for humans

Contact Info

Product

Resources

About