Reconstructing Spatio-Temporal Trajectories of Visual Object Memories in the Human Brain

Abstract

Our understanding of how information unfolds when we recall events from memory remains limited. In this study, we investigate whether the reconstruction of visual object memories follows a backward trajectory along the ventral visual stream with respect to perception, such that their neural feature representations are gradually reinstated from late areas close to the hippocampus backwards to lower-level sensory areas. We use multivariate analyses of fMRI activation patterns to map the constituent features of the object memories onto the brain during retrieval, and EEG-fMRI fusion to track the temporal evolution of the reactivated patterns. Participants studied new associations between verbs and randomly paired object images in an encoding phase, and subsequently recalled the objects when presented with the corresponding verb cue. Decoding reactivated memory features from fMRI activity revealed that retrieval patterns were dominated by conceptual features, represented in comparatively late visual and parietal areas. Representational-similarity-based fusion then allowed us to map the EEG patterns that emerged at each given time point of a trial onto the spatially resolved fMRI patterns. This fusion suggests that memory reconstruction proceeds backwards along the ventral visual stream from anterior fronto-temporal to posterior occipital and parietal regions, in line with a semantic-to-perceptual gradient. A linear regression on the peak time points of reactivated brain regions statistically confirms that the temporal progression is reversed with respect to encoding. Together, the results shed light onto the spatio-temporal trajectories along which memories are reconstructed during associative retrieval, and which features of an image are reconstructed when in time and where in the brain.

Replay-triggered Brain-wide Activation in Humans

“…The mean text activation was shown on the left, localised mostly in temporal cortex, and anterior cingulate cortex (ACC). These results were consistent with previous studies (e.g., 93 ). Compared to the image activation, the contrast of specific text revealed fewer distinctive activation pattern (right panel).…”

Neural patterns of conscious visual awareness in the Riddoch syndrome

“…The fact that there were patterns in V5 is a strong indication that these trials were associated with a visual motion percept as previous work has shown that the visual category of a hallucination correlates with activity in the visual areas specialised for that type of content 37 . Additionally, the MOG and hippocampus have been implicated in imagery and the retrieval of visual perceptual content from recent memory 38,39 .…”