Dynamic categorization rules alter representations in human visual cortex

Abstract

Everyday visual search tasks require objects to be categorized according to behavioral goals. For example, when searching for an apple at the supermarket, one might first find the Granny Smith apples by separating all visible apples into the categories “green” and “non-green”. However, suddenly remembering that your family actually likes Fuji apples would necessitate reconfiguring the boundary to separate “red” from “red-yellow” objects. Despite this need for flexibility, prior research on categorization has largely focused on understanding neural changes related to overlearning a single category boundary that bifurcates an object space. At the same time, studies of feature-based attention have provided some insight into flexible selection of features, but have mainly focused on selection of a single, usually low-level, feature, which is rarely sufficient to capture the complexity of categorizing higher-dimensional object sets. Here we addressed these gaps by asking human participants to categorize novel shape stimuli according to different linear and non-linear boundaries, a task that requires dynamically reconfiguring selective attention to emphasize different sets of abstract features. Using fMRI and multivariate analyses of retinotopically-defined visual areas, we found that shape representations in visual cortex became more distinct across relevant category boundaries in a context-dependent manner, with the largest changes in discriminability observed for stimuli near the category boundary. Importantly, these attention-induced modulations were linked to categorization performance. Together, these findings demonstrate that adaptive attentional modulations can alter representations of abstract feature dimensions in visual cortex to optimize object separability based on currently relevant category boundaries.

Disinhibitory signaling enables flexible coding of top-down information

“…The possibility that aspects of complex computations, typically ascribed to higher cortical areas, may also occur in lower-level areas challenges the prevailing notion of the cognitive limitations of lower-level areas. Building in representational redundancies throughout cortical hierarchies allows for complex computations to occur in both lower sensory areas and higher-level areas [25, 41, 42]. This mechanism enhances the robustness of representations and facilitates efficient processing, enabled by top-down feedback signals guiding multiplexing in sensory areas.…”

“…However, the extent to which lower cortical areas contribute to this adaptive process is not well understood. Past works have shown that responses within sensory areas could be modulated by top-down signals such as selective attention [16–25], and that non-sensory information such as memory content can be decoded from activity within sensory areas [26–29]. Yet, it remains poorly understood how these neural codes complement computations that take place in higher cortical areas.…”

Attentional guidance through object associations in visual cortex

“…Of note, for all analyses, we did not observe generalization between visually evoked responses in this localizer and the preview trials of the main task. This may be due to insufficient statistical power to detect such a generalization or that different features were relevant to discriminate objects in the periphery for quick guidance of eye movements and comparing objects to their memorized size in this localizer task, resulting in less overlap across tasks (Henderson et al, 2023; McKee et al, 2014). Another possibility is that preparatory activity in our task was overall less sensory-like (Gong et al, 2022).…”