Restricting visual exploration directly impedes neural activity, functional connectivity, and memory

Abstract

We move our eyes to explore the visual world, extract information, and create memories.The number of gaze fixations -the stops that the eyes make -has been shown to correlate with activity in the hippocampus, a region critical for memory, and with later recognition memory.Here, we combined eyetracking with fMRI to provide direct evidence for the relationships between gaze fixations, neural activity, and memory during scene viewing. Compared to free viewing, fixating a single location reduced: 1) subsequent memory, 2) neural activity along the ventral visual stream into the hippocampus, 3) neural similarity between effects of subsequent memory and visual exploration, and 4) functional connectivity among the hippocampus, parahippocampal place area, and other cortical regions. Gaze fixations were uniquely related to hippocampal activity, even after controlling for neural effects due to subsequent memory.Individual gaze fixations may provide the basic unit of information on which memory binding processes operate.

Neural correlates of subsequent memory-related gaze reinstatement

“…This necessarily sequential process of selecting visual features for fixation and further processing has important implications for memory. Research using eye movement monitoring indicates that during visual exploration, fixations and saccades support the binding of salient visual features and the relations among them into coherent and lasting memory traces (e.g., Liu, Shen, Olsen, & Ryan, 2017; Liu, Rosenbaum, & Ryan, 2020; for review, see Wynn, Shen, & Ryan, 2019). Moreover, such memory traces may be stored and subsequently recapitulated as patterns of eye movements or ‘scanpaths’ at retrieval (Noton & Stark, 1971b, 1971a; for review, see Wynn et al, 2019).…”

“…Consistent with this proposal, research using eye movement monitoring and neuroimaging techniques has established a critical role for eye movements in memory encoding (for review, see Meister & Buffalo, 2016; Ryan, Shen, & Liu, 2020). For example, at the behavioral level, recognition memory accuracy is significantly attenuated when eye movements during encoding are restricted (e.g., to a central fixation cross) as opposed to free (e.g., Damiano & Walther, 2019; Henderson, Williams, & Falk, 2005; Liu et al, 2020). At the neural level, restricting viewing to a fixed location during encoding results in attenuated activity in brain regions associated with memory and scene processing including the hippocampus (HPC) and parahippocampal place area (PPA), as well as reduced functional connectivity between these regions and other cortical regions (Liu, Rosenbaum, & Ryan, 2020).…”

“…For example, at the behavioral level, recognition memory accuracy is significantly attenuated when eye movements during encoding are restricted (e.g., to a central fixation cross) as opposed to free (e.g., Damiano & Walther, 2019; Henderson, Williams, & Falk, 2005; Liu et al, 2020). At the neural level, restricting viewing to a fixed location during encoding results in attenuated activity in brain regions associated with memory and scene processing including the hippocampus (HPC) and parahippocampal place area (PPA), as well as reduced functional connectivity between these regions and other cortical regions (Liu, Rosenbaum, & Ryan, 2020). When participants are free to explore, however, the number of fixations executed is positively predictive of subsequent memory performance (e.g., Liu et al, 2017; Loftus, 1972; Olsen et al, 2016) and of activity in the HPC (Liu et al, 2017, 2020; see also, Olsen et al, 2016), suggesting that eye movements are critically involved in the accumulation and encoding of visual feature information into lasting memory traces.…”

“…While there is now some evidence that mnemonic retrieval processes support gaze reinstatement at the neural level, the relationship between gaze reinstatement and encoding-related neural activity has yet to be investigated. Accordingly, the present study used the data from Liu et al (2020), in which gaze fixations during encoding of freely viewed scenes predicted both subsequent recognition memory and activity in the HPC and PPA, to elucidate the encoding mechanisms that support the formation and subsequent recapitulation of functional scanpaths. To this end, we first computed the spatial similarity between encoding and retrieval scanpaths and used this measure to predict recognition memory accuracy.…”

“…Given that prior work has revealed a significant role for top-down features (e.g., meaning, Henderson & Hayes, 2018; scene content, O’Connell & Walther, 2015) in guiding eye movements, above and beyond bottom-up image features (e.g., luminance, contrast, Itti & Koch, 2000), we hypothesized that gaze reinstatement would be related particularly to the viewing of informationally salient image regions. Finally, to uncover the neural correlates of functional gaze reinstatement, we analyzed neural activity patterns at encoding, both across the whole-brain and in memory-related regions of interest (i.e., HPC, PPA, see Liu et al, 2020), to identify brain regions that (1) predicted subsequent gaze reinstatement at retrieval, and (2) showed overlapping subsequent gaze reinstatement and subsequent memory effects. Given that previous work has linked gaze scanpaths, as a critical component of mnemonic representations, to successful encoding and retrieval, we hypothesized that functional gaze reinstatement would be supported by encoding-related neural activity in brain regions associated with visual processing (i.e., ventral visual stream regions) and memory (i.e., medial temporal lobe regions).…”