Shared Neural Mechanisms of Visual Perception and Imagery

Abstract

For decades, the extent to which visual imagery relies on similar neural mechanisms as visual perception has been a topic of debate. Here, we review recent neuroimaging studies comparing these two forms of visual experience. Their results suggest that there is large overlap in neural processing during perception and imagery: neural representations of imagined and perceived stimuli are similar in visual, parietal and frontal cortex. Furthermore, perception and imagery seem to rely on similar top-down connectivity. The most prominent difference is the absence of bottom-up processing during imagery. These findings fit well with the idea that imagery and perception rely on similar emulation or prediction processes. Externally and internally generated visual experience A large part of our sensory experience is visual. When walking down the street, we are bombarded with different colors, shapes and textures. Also when thinking about future or past events, most people tend to experience a rapid stream of detailed images [1]. Visual experience can be triggered externally, by events in the outside world that change the light that falls unto our retinas, such as during perception (see Glossary); or internally, by information from memory via a process known as mental imagery (see Box 1 on the relationship between imagery and working memory). Generally, these are seen as two distinct phenomena. However, they are phenomenologically similar which can sometimes lead us to question whether we really saw something or whether it was just our imagination. The question to what extent visual imagery relies on the same neural mechanisms as perception has been a topic of debate for decades. Originally, the debate was centered around the question whether imagery, like perception, relied on depictive, picture-like representations, or on symbolic, language-like representations [2-5]. Due to imagery's inherently private nature, for a long time it was impossible to address this question. Neuroimaging studies on the involvement of the primary visual cortex during imagery have now largely resolved this debate in favor of the depictive view [6]. However, a broader perspective, addressing the involvement and interaction of brain regions beyond the primary visual cortex, has been missing. The current review explores to what extent externally and internally generated visual experiences rely on similar neural mechanisms. We discuss the findings with respect to visual areas which are important in the depictivism-versus-symbolism debate, but we also focus on the involvement of parietal and frontal areas. Next, we focus on the temporal dynamics of neural processing during both forms of visual experience. After that, we discuss the overlap in directional connectivity between perception and imagery. We finish by concluding that perception and imagery are in fact highly similar and we discuss the issues and questions raised by this conclusion.

Neural dynamics of perceptual inference and its reversal during imagery

“…Specifically, all perceptual feedforward sweeps showed a negative relationship within the same imagery time window and all perceptual feedback sweeps showed a positive relationship within that same imagery time window, suggesting that the imagery feedback flow contained the complete stimulus representation that was inferred during perception. This result fits with the idea that imagery uses the same predictive processes that underlie perceptual inference to run off-line simulations of sensory representations under different hypotheses (11,(37)(38)(39)(40). In contrast to perception, during imagery, there is no bottom-up sensory input prompting hypothesis updating.…”

“…During mental imagery, information from memory is used to generate rich visual representations. Neural representations activated during imagery are highly similar to those activated during perception (11). Imagining an object activates similar object representations in high-level visual cortex (12)(13)(14)(15) and generating a mental image with simple visual features such as oriented gratings or letters is associated with perception-like activation of low-level visual areas (16)(17)(18).…”

Disentangling neocortical alpha/beta and hippocampal theta/gamma oscillations in human episodic memory formation

“…However, it seems plausible to suggest these cognitive processes are not completely restricted to their respective windows. Indeed, one may anticipate that some processing of the stimulus arises during mental association of the stimuli (Dijkstra, Bosch, & van Gerven, 2019), and that some mnemonic binding arises during sequence perception (Griffiths & Fuentemilla, 2019;Heusser et al, 2016). We do not dispute these ideas, but do suggest that any information representation that does arise during the binding window will be substantially smaller than the representation occurring during stimulus presentation, as we anticipate that the vast majority of stimulus processing to occur when participants are first shown the stimulus.…”

Visual mental imagery engages the left fusiform gyrus, but not the early visual cortex: a meta-analysis of neuroimaging evidence

“…The dominant model of VMI stipulates a functional and anatomical equivalence between VMI and visual perception 56 . According to this view, VMI and visual perception should rely upon the same cortical areas across the ventral cortical visual stream 18,56 . Progression of visual information would occur in a bottom-up fashion during perception, and in a reversed, top-down direction during VMI.…”

“…The dominant model of VMI 3,18 stipulates the existence of common neural substrates underlying VMI and visual perception, spanning across the ventral cortical visual stream.…”