Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex

Abstract

Early visual areas contain specific information about visual items maintained in working memory, suggesting a role for early visual cortex in more complex cognitive functions [1-4]. It is an open question, however, whether these areas also underlie the ability to internally generate images de novo (i.e., mental imagery). Research on mental imagery has to this point focused mostly on whether mental images activate early sensory areas, with mixed results [5-7]. Recent studies suggest that multivariate pattern analysis of neural activity patterns in visual regions can reveal content-specific representations during cognitive processes, even though overall activation levels are low [1-4]. Here, we used this approach [8, 9] to study item-specific activity patterns in early visual areas (V1-V3) when these items are internally generated. We could reliably decode stimulus identity from neural activity patterns in early visual cortex during both working memory and mental imagery. Crucially, these activity patterns resembled those evoked by bottom-up visual stimulation, suggesting that mental images are indeed "perception-like" in nature. These findings suggest that the visual cortex serves as a dynamic "blackboard" [10, 11] that is used during both bottom-up stimulus processing and top-down internal generation of mental content.

Laminar fMRI: Applications for cognitive neuroscience

“…These hypotheses are supported by brain decoding studies, which show that patterns of activity across early visual cortex during the maintenance of a previously presented grating stimulus (Harrison & Tong, 2009;Serences et al, 2009) or an internally imagined one (Albers et al, 2013) are similar to patterns induced by bottom-up perception of the same stimulus. Moreover, in the study by (Albers et al, 2013) a decoder trained on patterns induced during memory was equally accurate in decoding the orientation of an imagined grating as a classifier trained on patterns induced by perception, implying that representations of memory and imagery may be formed by the same perceptual mechanisms (Tong, 2013).…”

“…Laminar fMRI can be used to extend these findings to humans, revealing which layers contribute to successful decoding of visual stimuli during working memory and mental imagery. In addition, the hypothesis that working memory and mental imagery operate via the same mechanisms (Albers et al, 2013;Tong, 2013) could be explicitly tested, as it predicts that the laminar profiles of working memory and imagery responses should be the same. Some of the authors (SJDL and FPdL) are actively pursuing this research question at present.…”

Reinstatement of Associative Memories in Early Visual Cortex Is Signaled by the Hippocampus

“…The neural representations of this top-down working memory process were shown to be similar to those during bottom-up, passive viewing of the stimuli (Harrison and Tong, 2009;Serences et al, 2009;Xing et al, 2013). Shared neural representations were also found for perception and imagery, throughout the higher visual hierarchy (Kosslyn et al, 1995;Stokes et al, 2009;Reddy et al, 2010;Cichy et al, 2012;Lee et al, 2012), and recently in early visual cortex (Albers et al, 2013). The converging evidence of shared neural representations between perception and working memory (Harrison and Tong, 2009;Xing et al, 2013), imagery (Reddy et al, 2010;Cichy et al, 2012;Albers et al, 2013), and memory reinstatement suggest that these processes might be implemented in early visual cortex in a very similar manner (Tong, 2013) and may support conscious retrieval of memories (Slotnick and Schacter, 2006;Thakral et al, 2013).…”

“…Most studies on cortical reinstatement pooled recall stimuli of one category (e.g., faces or objects) to assess categoryspecific reinstatement (Polyn et al, 2005;Lewis-Peacock and Postle, 2008;Gordon et al, 2013;Liang et al, 2013). Some studies show evidence for memory-specific episodic reinstatement of different videos/pictures in the medial temporal lobe, specifically the hippocampus (Chadwick et al, 2010) and parahippocampal cortex (Staresina et al, 2012). Here, we extend these findings by showing that early visual cortex can also support stimulus-specific reinstatement, providing evidence for the predicted specificity of reinstatement even at the lowest levels of the sensory hierarchy.…”

Saccade Planning Evokes Topographically Specific Activity in the Dorsal and Ventral Streams

“…Here we test whether this critical finding also occurs in humans. To date, neuroimaging studies have almost unanimously reported that activity in human early visual cortex does not persist above pretrial baseline levels when maintaining feature information across working memory delays (Ester et al, 2009;Offen et al, 2009;Serences et al, 2009;Riggall and Postle, 2012;Albers et al, 2013).…”