Different states of priority recruit different neural representations in visual working memory

Abstract

We used functional magnetic resonance imaging (fMRI) to investigate the neural codes for representing stimulus information held in different states of priority in working memory. Human participants (male and female) performed delayed recall for 2 oriented gratings that could appear in any of several locations. Priority status was manipulated by a retrocue, such that one became the prioritized memory item (PMI) and another the unprioritized memory item (UMI). Using inverted encoding models (IEMs), we found that, in early visual cortex, the orientation of the UMI was represented in a neural representation that was rotated relative to the PMI. In intraparietal sulcus (IPS), we observed the analogous effect for the representation of the location of the UMI. Taken together, these results provide evidence for a common remapping mechanism that may be responsible for representing stimulus identity and stimulus context with different levels of priority in working memory.

Tracking stimulus representation across a 2-back visual working memory task

“…If this were not true, testing with the early-delay IEM would fail. For this reason, we believe that 'priority-based remapping' is a better characterization of these results (as well as those of [8,9]): the neural code is the same, but the mappings between specific stimulus values and specific neural patterns have changed. Stated another way, when a stimulus transitions to an unprioritized status, the set of mappings between stimulus values and neural patterns rotates such that the individual mappings are now different, but the distance (in orientation) between neural patterns, and therefore the neural code, is preserved [9].…”

“…This finding is reminiscent of the fMRI finding from van Loon et al [8], in which the representation of the category of the UMI in ventral temporal cortex projected into the opposite location of the MDS space relative to when it was the PMI. In addition, Yu, Teng & Postle [9], in an fMRI study of delayed serial retrocueing (DSR) with oriented gratings, have found IEM evidence for an active-but-opposite representation of the UMI in early visual cortex-i.e. IEM reconstructions of the orientation of the UMI are negatively correlated with the basis function that was used to train the model.…”

“…Thus, the results of both van Loon et al [8] and Yu, Teng & Postle. [9] suggest that the brain may use a recoding strategy to retain unprioritized information in working memory. Furthermore, this possibility is supported by the recent work on macaque neurophysiology, suggesting that neurons in the prefrontal cortex might 'morph population codes' in the face of a distractor to preserve information in a spatial working memory task [10,11].…”

Concurrent visual working memory bias in sequential integration of approximate number

“…Our findings appear consistent with the proposal that these networks might be relied on especially in unattended WM storage, while attended storage may more strongly engage sensory cortices 26 . However, new evidence exists that also sensory areas could be equipped with machinery to maintain representations in interference-proof formats 71 , potentially even during periods of inattention 27,72,73 . A latent persistence of concrete visual information in such formats may add to the high precision of WM recall we observed after attention was temporarily withdrawn.…”

“…A latent persistence of concrete visual information in such formats may add to the high precision of WM recall we observed after attention was temporarily withdrawn. For instance, recent fMRI evidence suggests that attended and unattended WM information could be represented in opposite patterns within shared content-coding areas 27,73 . Here, in terms of interactions with perception, we observed no behavioral indices of repulsion or suppression, but a pattern of attraction biases consistent with crosstalk at different levels of the cortical processing hierarchy (but see 74 for evidence that behavioral attraction and neural repulsion are not mutually exclusive).…”