Temporal Dynamics of Competition between Statistical Learning and Episodic Memory in Intracranial Recordings of Human Visual Cortex

Abstract

The function of long-term memory is not just to reminisce about the past, but also to make predictions that help us behave appropriately and efficiently in the future. This predictive function of memory provides a new perspective on the classic question from memory research of why we remember some things but not others. If prediction is a key outcome of memory, then the extent to which an item generates a prediction signifies that this information already exists in memory and need not be encoded. We tested this principle using human intracranial EEG as a time-resolved method to quantify prediction in visual cortex during a statistical learning task and link the strength of these predictions to subsequent episodic memory behavior. Epilepsy patients of both sexes viewed rapid streams of scenes, some of which contained regularities that allowed the category of the next scene to be predicted. We verified that statistical learning occurred using neural frequency tagging and measured category prediction with multivariate pattern analysis. Although neural prediction was robust overall, this was driven entirely by predictive items that were subsequently forgotten. Such interference provides a mechanism by which prediction can regulate memory formation to prioritize encoding of information that could help learn new predictive relationships.SIGNIFICANCE STATEMENTWhen faced with a new experience, we are rarely at a loss for what to do. Rather, because many aspects of the world are stable over time, we rely on past experiences to generate expectations that guide behavior. Here we show that these expectations during a new experience come at the expense of memory for that experience. From intracranial recordings of visual cortex, we decoded what humans expected to see next in a series of photographs based on patterns of neural activity. Photographs that generated strong neural expectations were more likely to be forgotten in a later behavioral memory test. Prioritizing the storage of experiences that currently lead to weak expectations could help improve these expectations in future encounters.

Online abstraction during statistical learning revealed by neural entrainment from intracranial recordings

“…We tested 8 patients (1 female; age range: 21-61; mean age = 37.8) who had been surgically implanted with intracranial electrodes for localization of seizure onset zone (see Table 1 for patient demographics and details on implant). This sample size was chosen a priori based on Sherman et al (2022 ). Two patients were tested a second time (two days later) because their first dataset was found to be unusable: one of these patients experienced severe eye irritation during the first testing session and there was a technical error with the triggers for the other patient.…”

“…Given prior evidence for entrainment in sensory regions, we were interested in measuring neural responses in visual regions. We constructed a broad visual cortex region of interest (ROI), as in Sherman et al (2022 ), on the Montreal Neurological Institute (MNI) T1 2mm standard brain by combining the Occipital Lobe ROI from the MNI Structural Atlas and the following ROIs from the Harvard-Oxford Cortical Structural Atlas: Inferior Temporal Gyrus (temporo-ocipital part), Lateral Occipital Cortex (superior division), Lateral Occipital Cortex (inferior division), Intracalcarine Cortex, Cuneal Cortex, Parahippocampal Gyrus (posterior division), Lingual Gyrus, Temporal Occipital Fusiform Cortex, Occipital Fusiform Gyrus, Supracalcarine Cortex, and Occipital Pole. Each ROI was thresholded at 10% and then concatenated to create a single mask of visual cortex.…”

“…Thus, we combine, for the first time, the method of neural entrainment as an online measure with a task design optimized for evaluating categorical abstraction during statistical learning. This task builds on our recent intracranial EEG (iEEG) study that conflated regularities at the exemplar and category level ( Sherman et al, 2022 ). Here we evaluate these two levels of abstraction separately in distinct conditions (relative to a random baseline condition), allowing us to quantify neural entrainment online during exemplar-level and category-level statistical learning.…”

“…Given prior work demonstrating neural entrainment in sensory regions(Henin et al, 2021;Sherman et al, 2022), we focused our analyses on visual cortex. Specifically, we computed coherence within each contact localized to the visual cortex ROI (116 contacts) and averaged the coherence across contacts, within each participant.…”

A Generative Model of Memory Construction and Consolidation

“…During perception, the generative model provides an ongoing estimate of novelty from its reconstruction error. Memories that are very consistent with previous experience do not need to be encoded in detail in the autoassociative ‘teacher’ network (see Bein et al, 2021; Biderman et al, 2020; Schacter et al, 2007; Sherman et al, 2022). (How the model deals with events composed of both predictable and unpredictable elements is addressed in the next section.…”