Processing multiple visual objects is limited by overlap in neural channels

Abstract

High-level visual categories (e.g., faces, bodies, scenes, and objects) have separable neural representations across the visual cortex. Here, we show that this division of neural resources affects the ability to simultaneously process multiple items. In a behavioral task, we found that performance was superior when items were drawn from different categories (e.g., two faces/two scenes) compared to when items were drawn from one category (e.g., four faces). The magnitude of this mixed-category benefit depended on which stimulus categories were paired together (e.g., faces and scenes showed a greater behavioral benefit than objects and scenes). Using functional neuroimaging (i.e., functional MRI), we showed that the size of the mixed-category benefit was predicted by the amount of separation between neural response patterns, particularly within occipitotemporal cortex. These results suggest that the ability to process multiple items at once is limited by the extent to which those items are represented by separate neural populations.working memory | capacity limitations | representational similarity | competition | visual cognition A n influential idea in neuroscience is that there is an intrinsic relationship between cognitive capacity and neural organization. For example, seminal cognitive models claim there are distinct resources devoted to perceiving and remembering auditory and visual information (1, 2). This cognitive distinction is reflected in the separate cortical regions devoted to processing sensory information from each modality (3). Similarly, within the domain of vision, when items are placed near each other, they interfere more than when they are spaced farther apart (4, 5). These behavioral effects have been linked to receptive fields and the retinotopic organization of early visual areas, in which items that are farther apart activate more separable neural populations (6-8). Thus, there are multiple cognitive domains in which it has been proposed that capacity limitations in behavior are intrinsically driven by competition for representation at the neural level (4, 7-10).However, in the realm of high-level vision, evidence linking neural organization to behavioral capacities is sparse, although neural findings suggest there may be opportunities for such a link. For example, results from functional MRI (fMRI) and single-unit recording have found distinct clusters of neurons that selectively respond to categories such as faces, bodies, scenes, and objects (11, 12). These categories also elicit distinctive activation patterns across the ventral stream as measured with fMRI (13,14). Together, these results raise the interesting possibility that there are partially separate cognitive resources available for processing different object categories.In contrast, many prominent theories of visual cognition do not consider the possibility that different categories are processed by different representational mechanisms. For example, most models of attention and working memory assume or imply that these process...

Similarity effects in visual working memory

“…This finding is consistent with those from verbal working memory, which show better memory for phonologically dissimilar than similar words (Baddeley, 1966). They support theories such as the multiple-resource theory (Olson & Jiang, 2002;Wheeler & Treisman, 2002) or its neural instantiation -the cortical resource theory (Cohen et al, 2014). Dissimilar stimuli yield better memory because peaks of neural activation for these stimuli are widely separated, minimizing interference.…”

“…One pattern of data suggests that similarity is detrimental. For example, memory for an array of faces and scenes is better than memory for just faces or scenes (Cohen, Konkle, Rhee, Nakayama, & Alvarez, 2014). This finding is consistent with those from verbal working memory, which show better memory for phonologically dissimilar than similar words (Baddeley, 1966).…”

Is VIRTU4L larger than VIR7UAL? Automatic processing of number quantity and lexical representations in leet words.

“…Alternatively, one may argue that all dimensions of the stimuli are processed, so that the digits embedded in leet words are processed not only as letters but also as numerical quantities. Indeed, a positive answer to this question would strongly suggest that, upon presentation of visual stimuli, there is activation from multiple codes (quantities, lexical representations) in the cognitive system (see Cohen et al, 2014). To tease apart these two explanations, we employed a physical-size judgment tasks with leet words.…”

“…Importantly, there are proposals that relate the ability of our cognitive system to process multiple representations of a given stimulus at once with the neural overlap between the categories of the items presented. In an experiment using functional neuroimaging, Cohen et al (2014) found that the ability to process stimuli from different categories (e.g., faces vs scenes) in a single presentation was predicted by the amount of separation between neural response patterns, particularly within occipitotemporal cortex. The idea was that neural overlap would imply sharing resources whereas non-overlapping neural representations would imply separate representational resources (Cohen et al, 2014).…”

“…In an experiment using functional neuroimaging, Cohen et al (2014) found that the ability to process stimuli from different categories (e.g., faces vs scenes) in a single presentation was predicted by the amount of separation between neural response patterns, particularly within occipitotemporal cortex. The idea was that neural overlap would imply sharing resources whereas non-overlapping neural representations would imply separate representational resources (Cohen et al, 2014). In the present experiment, one can formulate a parallel explanation: When presented with leet words, letters are normalized in the "visual word-form" area (Cohen et al, 2000, Polk et al, 2002, whereas digits would be (independently) recognized in the inferior temporal gyrus (the "visual number-form" area; see Shum et al, 2013).…”