Color Tuning of Face-Selective Neurons in Macaque Inferior Temporal Cortex

Abstract

What role does color play in the neural representation of complex shapes? We approached the question by measuring color responses of face-selective neurons, using fMRI-guided microelectrode recording of the middle and anterior face patches of inferior temporal cortex (IT) in rhesus macaques. Face-selective cells responded weakly to pure color (equiluminant) photographs of faces. But many of the cells nonetheless showed a bias for warm colors when assessed using images that preserved the luminance contrast relationships of the original photographs. This bias was also found for non-face-selective neurons. Fourier analysis uncovered two components: the first harmonic, accounting for most of the tuning, was biased toward reddish colors, corresponding to the L>M pole of the L-M cardinal axis. The second harmonic showed a bias for modulation between blue and yellow colors axis, corresponding to the S-cone axis. To test what role face-selective cells play in behavior, we related the information content of the neural population with the distribution of face colors. The analyses show that face-selective cells are not optimally tuned to discriminate face colors, but are consistent with the idea that face-selective cells contribute selectively to processing the green-red contrast of faces. The research supports the hypothesis that color-specific information related to the discrimination of objects, including faces, is handled by neural circuits that are independent of shape-selective cortex, as captured by the multistage parallel processing framework of IT ( Lafer-Sousa and Conway, 2013 ).

Temporal dynamics of the neural representation of hue and luminance polarity

“…The magnitude of the CSD was different among the functional regions identified in the Ventral Visual Pathway (p=0.002, repeated measures oneway ANOVA): the color-biased regions were not different from face-biased regions (p=0.12; paired two sided t-test, uncorrected); but were different from LO (p=0.01), and from place-biased regions (p=0.02) (Figure 8c). These results provide a direct measure of neural activity, and confirm the indirect measurements obtained with fMRI suggesting that fMRI-identified color-biased regions (and possibly facebiased regions) play an important role in color processing 75 .…”

“…That it was possible to decode hue generalizing across luminance polarity supports the hypothesis that the brain has a representation of hue that is separate from the representation of luminance contrast. The latency at which decoding became significant was 115 ms [35,355] for the generalization problems, and 80 ms [75,85] for the identity problems (the latency of the generalization problems was greater than the upper 95% CI limit for the latency of the identity problems, but a direct comparison of the bootstrapped values was not significant, p=0.09). As with the luminance-polarity decoding problems, decoding had a higher peak magnitude for the identity problems (78% [74,82]) compared to the generalization problems (56% [53,59]), which provides support for the hypothesis that the brain also has a representation of hue that is inseparable from the representation of luminance contrast.…”

“…The time to peak was the same for the identity and generalization problems (identity problems: solid vertical line, 121.8 ms [120, 130]; generalization problems: dashed vertical line, 119.1 ms [115,130]). Onset of significant decoding for the identity problems was achieved at 80 ms [75,85], and for the generalization problems, at 115 [35,355]. Other conventions as for panel (b).…”

Face detection in untrained deep neural networks

“…1f , n untrained = 465, n monkey = 158, NS, two-sided rank-sum test, P = 7.69 × 10 −2 , r rbc = 9.25 × 10 −2 , two-sided Kolmogorov–Smirnov test, P = 2.49 × 10 −4 , d = 2.32 × 10 −2 ) and a significantly higher value than those measured from a shuffled response (Fig. 1f , n untrained = 465, n shuffled = 465, two-sided rank-sum test, P = 1.49 × 10 −25 , r rbc = 5.09 × 10 −1 ) for various definitions of the FSI 5 , 17 , 60 (Supplementary Fig. 3 ).…”

Representation of color, form, and their conjunction across the human ventral visual pathway

“…However, color and form information may be encoded in distributed, fine-grained activation patterns that univariate mean-activation methods cannot detect (e.g., Haxby et al, 2001 ). Indeed, macaque IT and color regions contain both color and form information ( Komatsu and Ideura, 1993 ; McMahon and Olson, 2009 ; Chang et al, 2017 ; Rosenthal et al, 2018 ; Duyck et al., 2021 ) and the human shape-selective region in lateral occipital cortex has been found to contain color information using fMRI multivoxel pattern analysis ( Bannert and Bartels, 2013 , 2018 ).…”