Spatial Attention Enhances Crowded Stimulus Encoding Across Modeled Receptive Fields by Increasing Redundancy of Feature Representations

Abstract

Any visual system, biological or artificial, must make a trade-off between the number of units used to represent the visual environment and the spatial resolution of the sampling array. Humans and some other animals are able to allocate attention to spatial locations to reconfigure the sampling array of receptive fields (RFs), thereby enhancing the spatial resolution of representations without changing the overall number of sampling units. Here, we examine how representations of visual features in a fully convolutional neural network interact and interfere with each other in an eccentricity-dependent RF pooling array and how these interactions are influenced by dynamic changes in spatial resolution across the array. We study these feature interactions within the framework of visual crowding, a well-characterized perceptual phenomenon in which target objects in the visual periphery that are easily identified in isolation are much more difficult to identify when flanked by similar nearby objects. By separately simulating effects of spatial attention on RF size and on the density of the pooling array, we demonstrate that the increase in RF density due to attention is more beneficial than changes in RF size for enhancing target classification for crowded stimuli. Furthermore, by varying target and flanker spacing, as well as the spatial extent of attention, we find that feature redundancy across RFs has more influence on target classification than the fidelity of the feature representations themselves. Based on these findings, we propose a candidate mechanism by which spatial attention relieves visual crowding through enhanced feature redundancy that is mostly due to increased RF density.

Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model

“…Our modeling approach allowed us to examine the theoretic impact of each change that is associated with gain systematically and quantify the potential benefit to detection and discrimination task performance. At larger scales or in other tasks there are theoretical reasons to expect that task performance will improve due to these effects, (Kay et al, 2015;Vo et al, 2017;Theiss et al, 2022) Whether our conclusions can generalize to the behavior of attentional gain in biological neural circuits is limited both by how well the neural network observer model approximates the functioning of those neural circuits and by the model's ability to predict behavior. There are several reasons to suggest that the model captures relevant properties of both object recognition and the primate visual system.…”

“…Our modeling approach allowed us to examine the theoretic impact of each change that is associated with gain systematically and quantify the potential benefit to detection and discrimination task performance. At larger scales or in other tasks there are theoretical reasons to expect that task performance will improve due to these effects, ( Kay et al, 2015 ; Vo et al, 2017 ; Theiss et al, 2022 )…”

Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model

“…Our modeling approach allowed us to examine the theoretical impact of each change associated with gain systematically and quantify that these provided no benefit to detection or discrimination task performance. Nevertheless, previous modeling work has demonstrated that shift and shrinkage, in particular, can increase the resolution and redundancy of receptive field coverage ( Kay et al, 2015 ; Vo et al, 2017 ; Theiss et al, 2022 ). Our results differ along two important dimensions: first, we scaled the magnitude of shift, shrinkage, and sensitivity changes to those induced by Gaussian gain (at a gain strength that was matched to human performance) and these were in general equal to or smaller in magnitude to what was observed in these previous papers.…”

Effects of involuntary and voluntary attention on critical spacing of visual crowding

“…Furthermore, RFs in humans and other animals have been observed to both shrink in size and shift toward the locus of attention with sustained spatial attention ( Womelsdorf, Anton-Erxleben, Pieper, & Treue, 2006 ; Klein, Harvey, & Dumoulin, 2014 ). Using computational modeling approaches, Baruch and Yeshurun (2014) showed that this reconfiguration of RFs with attention could explain a number of attentional effects on neural responses, and Theiss, Bowen, and Silver (2021) showed that a similar mechanism could be implemented in a convolutional neural network, resulting in attentional enhancement of performance on a visual crowding task. Additionally, He, Wang, and Fang (2019) showed that, after perceptual learning of a crowded orientation discrimination task, decreases in RF size of individual fMRI voxels in cortical area V2 correlated with improved performance that resulted from perceptual learning.…”

“…Crowding has been modeled as arising from inherent limits in the size and density of cortical receptive fields (RFs) in the visual periphery, especially when compared with central vision ( Parkes, Lund, Angelucci, Solomon, & Morgan, 2001 ; Balas, Nakano, & Rosenholtz, 2009 ; Dakin, Cass, Greenwood, & Bex, 2010 ; Greenwood, Bex, & Dakin, 2010 ; Freeman & Simoncelli, 2011 ; Rosenholtz, 2016 ). One mechanism by which visual spatial attention might relieve crowding is by locally increasing the density of RFs that sample the target location ( Baruch & Yeshurun, 2014 ; Theiss, Bowen, & Silver, 2021 ). Neurophysiologically, it has been shown that sustained visual spatial attention causes RFs to shift toward the locus of attention and to shrink in size ( Womelsdorf, Anton-Erxleben, Pieper, & Treue, 2006 ; Klein, Harvey, & Dumoulin, 2014 ).…”