Network Anisotropy Trumps Noise for Efficient Object Coding in Macaque Inferior Temporal Cortex

Abstract

How neuronal ensembles compute information is actively studied in early visual cortex. Much less is known about how local ensembles function in inferior temporal (IT) cortex, the last stage of the ventral visual pathway that supports visual recognition. Previous reports suggested that nearby neurons carry information mostly independently, supporting efficient processing (Barlow, 1961). However, others postulate that noise covariation effects may depend on network anisotropy/homogeneity and on how the covariation relates to representation. Do slow trial-by-trial noise covariations increase or decrease IT's object coding capability, how does encoding capability relate to correlational structure (i.e., the spatial pattern of signal and noise redundancy/homogeneity across neurons), and does knowledge of correlational structure matter for decoding? We recorded simultaneously from ϳ80 spiking neurons in ϳ1 mm 3 of macaque IT under light neurolept anesthesia. Noise correlations were stronger for neurons with correlated tuning, and noise covariations reduced object encoding capability, including generalization across object pose and illumination. Knowledge of noise covariations did not lead to better decoding performance. However, knowledge of anisotropy/homogeneity improved encoding and decoding efficiency by reducing the number of neurons needed to reach a given performance level. Such correlated neurons were found mostly in supragranular and infragranular layers, supporting theories that link recurrent circuitry to manifold representation. These results suggest that redundancy benefits manifold learning of complex high-dimensional information and that subsets of neurons may be more immune to noise covariation than others.

Emergence of transformation-tolerant representations of visual objects in rat lateral extrastriate cortex

“…The second motivation was the need of excluding potential effects of top-down signals, task- and state-dependence, learning and memory ( Gavornik and Bear, 2014 ; Cooke and Bear, 2015 ; Burgess et al, 2016 ), which are all detrimental when the goal is to understand the initial, largely reflexive, feed-forward sweep of activation through a visual processing hierarchy ( DiCarlo et al, 2012 ). For these reasons, many primate studies have investigated ventral stream functions in anesthetized monkeys [e.g., see ( Kobatake and Tanaka, 1994 ; Ito et al, 1995 ; Logothetis et al, 1999 ; Tsunoda et al, 2001 ; Sato et al, 2013 ; Chen et al, 2015 )] or, if awake animals were used, under passive viewing conditions [e.g., see ( Pasupathy and Connor, 2002 ; Brincat and Connor, 2004 ; Hung et al, 2005 ; Kiani et al, 2007 ; Willmore et al, 2010 ; Rust and Dicarlo, 2010 ; Hong et al, 2016 ; El-Shamayleh and Pasupathy, 2016 )].…”

Encoding of Predictable and Unpredictable Stimuli by Inferior Temporal Cortical Neurons

“…Given that we did not measure the response covariance matrix and are ignorant about its readout, we cannot make claims about how well (i.e., quantitatively) the brain can decode the IT population responses. Chen, Lin, Hsu, & Hung (2015) assessed the effect of noise correlation in a small population of IT neurons (up to 87 neurons) on object decoding with a linear classifier. They found that noise correlations slightly decreased decoding, which is in line with their on average positive signal correlations.…”

Information-Limiting Correlations in Large Neural Populations

“…Measuring correlations alone does not show whether they impact information coding, because it is the relation between average population activity and the structure of the noise correlations that affects information. Noise correlations have been shown to increase (Romo et al, 2003;Zavitz et al, 2019), decrease (Chen et al, 2015;Graf and Andersen, 2015), or have minimal effect (Averbeck andLee, 2003, 2006; on information coding. Across these studies, measured effects have been modest, and they were estimated in small ensembles of at most 10 s of neurons.…”