Trade-Off between Object Selectivity and Tolerance in Monkey Inferotemporal Cortex

Abstract

Object recognition requires both selectivity among different objects and tolerance to vastly different retinal images of the same object, resulting from natural variation in (e.g.) position, size, illumination, and clutter. Thus, discovering neuronal responses that have object selectivity and tolerance to identity-preserving transformations is fundamental to understanding object recognition. Although selectivity and tolerance are found at the highest level of the primate ventral visual stream [the inferotemporal cortex (IT)], both properties are highly varied and poorly understood. If an IT neuron has very sharp selectivity for a unique combination of object features ("diagnostic features"), this might automatically endow it with high tolerance. However, this relationship cannot be taken as given; although some IT neurons are highly object selective and some are highly tolerant, the empirical connection of these key properties is unknown. In this study, we systematically measured both object selectivity and tolerance to different identity-preserving image transformations in the spiking responses of a population of monkey IT neurons. We found that IT neurons with high object selectivity typically have low tolerance (and vice versa), regardless of how object selectivity was quantified and the type of tolerance examined. The discovery of this trade-off illuminates object selectivity and tolerance in IT and unifies a range of previous, seemingly disparate results. This finding also argues against the idea that diagnostic conjunctions of features guarantee tolerance. Instead, it is naturally explained by object recognition models in which object selectivity is built through AND-like tuning mechanisms.

Contour Curvature As an Invariant Code for Objects in Visual Area V4

“…Nevertheless, stimulus size may also influence neuronal responses by providing an overall gain modulation. Consistent with previous reports in IT cortex (Ito et al, 1995;Tanaka, 1996;Zoccolan et al, 2007), we found that many V4 neurons of either type (ie, sensitive to absolute or normalized curvature) showed response gain modulation across transformations of scale and position (Fig. 3, Fig.…”

Elements for a Neural Theory of the Processing of Dynamic Faces

“…It also shows good agreement with other data in V4 on the tuning for two-bar stimuli and for boundary conformations (Pasupathy & Connor, 2001;Reynolds, Chelazzi, & Desimone, 1999). The IT-like units of the model exhibit selectivity and invariance that are very similar to those of IT neurons (Hung, Kreiman, Poggio, & DiCarlo, 2005) for the same set of stimuli, and the model helped explain the tradeo¤ between invariance and selectivity observed in the IT in the presence of clutter (Zoccolan, Kouh, Poggio, & DiCarlo, 2007). Also, the model accurately matches the psychophysical performance of human observers for rapid animal versus nonanimal recognition (Serre, Oliva, & Poggio, 2007), a task that is not likely to be strongly influenced by cortical backprojections.…”

Unwrapping the Ventral Stream: Figure 1.

“…IT was reliably able to decode object identity across a range of transformations, whereas the V4 population performed poorly. Although demonstrations of complex feature selectivity and invariant representations in V4 and IT already exist (Pasupathy and Connor, 2002;Zoccolan et al, 2007), these results provide the first systematic comparison of matched populations across the two different visual areas, documenting an increase in both selectivity and invariance.…”