Neural Representations Integrate the Current Field of View with the Remembered 360° Panorama in Scene-Selective Cortex

Abstract

We experience our visual environment as a seamless, immersive panorama. Yet, each view is discrete and fleeting, separated by expansive eye movements and discontinuous views of our spatial surroundings. How are discrete views of a panoramic environment knit together into a broad, unified memory representation? Regions of the brain's "scene network" are well poised to integrate retinal input and memory [1]: they are visually driven [2, 3] but also densely interconnected with memory structures in the medial temporal lobe [4]. Further, these regions harbor memory signals relevant for navigation [5-8] and adapt across overlapping shifts in scene viewpoint [9, 10]. However, it is unknown whether regions of the scene network support visual memory for the panoramic environment outside of the current field of view and, further, how memory for the surrounding environment influences ongoing perception. Here, we demonstrate that specific regions of the scene network-the retrosplenial complex (RSC) and occipital place area (OPA)-unite discrete views of a 360° panoramic environment, both current and out of sight, in a common representational space. Further, individual scene views prime associated representations of the panoramic environment in behavior, facilitating subsequent perceptual judgments. We propose that this dynamic interplay between memory and perception plays an important role in weaving the fabric of continuous visual experience.

Cortico-hippocampal network connections support the multidimensional quality of episodic memory

“…Much research has documented the complementary roles role of PHC and RSC in spatial processing and navigation (Epstein, 2008; Mitchell, Czajkowski, Zhang, Jeffery, & Nelson, 2018 , including sensitivity to distance within virtual environments (Sulpizio, Committeri, & Galati, 2014) . Moreover, a recent study showed that RSC is important for forming coherent scene representations similarly using 360° panorama scenes (Robertson, Hermann, Mynick, Kravitz, & Kanwisher, 2016) , in line with its role in viewpoint precision demonstrated here. Surprisingly, we found no evidence that hippocampal connectivity supported the precision of PM spatial representations, which is in contrast to evidence implicating the hippocampus, particularly posterior, in spatial precision specifically (Nadel, Hoscheidt, & Ryan, 2013; Nilakantan et al, 2017, 2018; Stevenson et al, 2018 .…”

Retrosplenial cortex and its role in spatial cognition

“…However, it was not demonstrated that subjects had been able to form global maps of the virtual space (that is, the reference frame in which the local spaces were set), so the question remains unanswered about whether RSC is also involved in encoding global space. Robertson et al (2016) also found encoding of local "landmarks" in a setting in which subjects viewed segments of a 360-degree panorama that either did or did not overlap.…”

“…Some meta-analyses of human imaging studies have indicated that higher RSC activation occurs when subjects process landmark information (Maguire, 2001b;Spiers and Maguire, 2006;Auger et al, 2012;Mullally et al, 2012;Auger and Maguire, 2013) and associate the current panoramic visual scene with memory (Robertson et al, 2016).…”

Progression from feature-specific brain activity to hippocampal binding during episodic encoding

“…Specifically, successful encoding of the object's color was associated with enhanced activity of right ventral visual cortical areas, whereas subsequent memory for the sound was positively related to activity in bilateral superior temporal (auditory) cortex. Neural correlates of successful scene encoding were more widespread, with positive changes in activity observed in bilateral dorsal medial parietal and occipital cortex, as well as bilateral retrosplenial and parahippocampal cortex, frequently reported to be sensitive to specific representations of spatial locations and perspectives (e.g., Epstein, 2008;Robertson et al, 2016;Robin et al, 2018;Berens et al, 2019). Thus, we were able to detect unique patterns of 15 brain activity that positively tracked different kinds of episodic features encoded simultaneously, with the most widespread effects for encoding scene details.…”