Joint coding of shape and blur in area V4

Abstract

Edge blur, a prevalent feature of natural images, is believed to facilitate multiple visual processes including segmentation and depth perception. Furthermore, image descriptions that explicitly combine blur and shape information provide complete representations of naturalistic scenes. Here we report the first demonstration of blur encoding in primate visual cortex: neurons in macaque V4 exhibit tuning for both object shape and boundary blur, with observed blur tuning not explained by potential confounds including stimulus size, intensity, or curvature. A descriptive model wherein blur selectivity is cast as a distinct neural process that modulates the gain of shape-selective V4 neurons explains observed data, supporting the hypothesis that shape and blur are fundamental features of a sufficient neural code for natural image representation in V4.

Artiphysiology reveals visual preferences underlying V4-like blur selectivity in a deep convolutional neural network

“…Relationship to neurophysiology . Our study was motivated by a previous report of boundary blur encoding in V4, and our results are consistent with those in V4 in terms of the approximate proportions of blur-selective units detected -roughly on the order of 20% of units (Oleskiw et al, 2018). To determine whether blur-encoding neurons in V4 map onto the subjective classes defined here may require experiments that use far more stimuli than possible given practical limitations.…”

“…To gain insight into the potential visual selectivity of units previously identified as encoding boundary blur in area V4 (Oleskiw et al, 2018), we adopted the visual stimulus set from the macaque experiments to identify, within a CNN, blur selective units from shallow to deep layers that could then be studied in depth. Major conclusions from our study are as follows: (1) there are a substantial fraction of units in the CNN that appear to be involved in blur encoding when tested with simple, artificial stimuli, (2) several subjective forms of blur selectivity emerged, including BCB units that are selective for boundaries defined by blur contrast, (3) blur selectivity is transferred and transformed across layers, (4) units selective for blank or relatively uniform image patches are also detected by our simple blur assay, and (5) many units involved in blur encoding are missed by our assay.…”

“…Visual stimuli . Our stimulus set was like that of Oleskiw et al (2018), which was based on the design of Pasupathy and Connor (2001). It was generated from 51 simple shapes ( Figure 1A) drawn at up to eight rotations each ( Figure 1B; 45 o increments), where rotations producing a replica of a previous rotation were omitted.…”

“…One recent study in the mid-level cortical area V4, part of the ventral, form-processing stream, found that some neurons believed to be tuned for the shape of object boundaries were most selective for intermediate levels of blur (Oleskiw et al, 2018). This study was carried out with simple shape stimuli that were blurred to varying degrees by Gaussian convolution, and thus it remains unknown what the selectivity of such blur-preferring neurons might be if tested with a broad range of natural images.…”

“…We therefore took the set of simple blurred stimuli used in the neurophysiological study of Oleskiw et al (2018) and presented them to the same CNN tested by Pospisil et al (2018) with the goal of determining (1) to what degree simple artificial stimuli are useful for detecting blur selective units in a deep network, and (2) what types of selectivity such units have when tested on natural scenes. We found that many units were identified as blur selective across all levels of the network and we defined a class of units that we refer to as blur-contrast boundary (BCB) units that respond best to oriented or shaped boundaries between regions of high and low blur levels.…”