Eye Movements as Proxy for Visual Working Memory Usage: Increased Reliance on the External World in Korsakoff Syndrome

Abstract

In the assessment of visual working memory, estimating the maximum capacity is currently the gold standard. However, traditional tasks disregard that information generally remains available in the external world. Only when to-be-used information is not readily accessible, memory is taxed. Otherwise, people sample information from the environment as a form of cognitive offloading. To investigate how memory deficits impact the trade-off between sampling externally or storing internally, we compared gaze behaviour of individuals with Korsakoff amnesia (n = 24, age range 47–74 years) and healthy controls (n = 27, age range 40–81 years) on a copy task that provoked different strategies by having information freely accessible (facilitating sampling) or introducing a gaze-contingent waiting time (provoking storing). Indeed, patients sampled more often and longer, compared to controls. When sampling became time-consuming, controls reduced sampling and memorised more. Patients also showed reduced and longer sampling in this condition, suggesting an attempt at memorisation. Importantly, however, patients sampled disproportionately more often than controls, whilst accuracy dropped. This finding suggests that amnesia patients sample frequently and do not fully compensate for increased sampling costs by memorising more at once. In other words, Korsakoff amnesia resulted in a heavy reliance on the world as ‘external memory’.

Sensory Input Matching Visual Working Memory Guides Internal Prioritization

“…Luck & Vogel, 1997; Vogel et al, 2006). A more recent line of work instead leaves to-be-memorized items externally available indefinitely to resemble more stable environments (Ballard et al, 1995; Böing et al, 2023; Chota et al, 2023; Draschkow et al, 2021; Grinschgl et al, 2021; Hoogerbrugge et al, 2023; Koevoet, Naber, Strauch, et al, 2023; Meyerhoff et al, 2021; O’Regan, 1992; Risko & Gilbert, 2016; Sahakian et al, 2023a, 2023b; Somai et al, 2020; Van der Stigchel, 2020). Here we combined these approaches by using sensory input which appeared, disappeared and reappeared within a matter of seconds to mimic a more dynamic daily-life situation (Figure 1).…”

“…If relevant information is available in the environment, participants tend to not load up VWM fully (e.g. Ballard et al, 1995;Böing et al, 2023;Draschkow et al, 2021;Hoogerbrugge et al, 2023;Kvitelashvili & Kessler, 2024;Somai et al, 2020) -…”

“…or sometimes not at all (Chota et al, 2023). Instead, participants often prefer to resample to-be-memorized material and only encode one or two items at a time (Aivar et al, 2005;Ballard et al, 1995;Böing et al, 2023;Chota et al, 2023;Draschkow et al, 2021;Droll & Hayhoe, 2007;Grinschgl et al, 2021;Hoogerbrugge et al, 2023;Kvitelashvili & Kessler, 2024;Melnik et al, 2018;Meyerhoff et al, 2021;O'Regan, 1992;Risko & Gilbert, 2016;Sahakian et al, 2023aSahakian et al, , 2023bSomai et al, 2020;Van der Stigchel, 2020;L. Xu et al, 2023).…”

Just-in-Time Encoding Into Visual Working Memory Is Contingent Upon Constant Availability of External Information

“…Therefore, VWM can be regarded as being part of a dynamic system that constantly weighs the costs of maintaining a (high) memory load against the costs of external sampling. Indeed, consistent with the example of a jigsaw puzzle, several studies have found participants to minimally utilize VWM in many circumstances where information could be retrieved just-in-time from the environment instead (Ballard et al, 1995;Böing et al, 2023;Draschkow et al, 2021;Droll & Hayhoe, 2007;Gajewski & Henderson, 2005;Gray et al, 2006;Hayhoe et al, 2003;Inamdar & Pomplun, 2003;Melnik et al, 2018;Risko & Dunn, 2015;Risko & Gilbert, 2016;Sahakian et al, 2023Sahakian et al, , 2024Somai et al, 2020;Triesch et al, 2003). In this just-in-time approach, external information is only fixated and encoded into memory if and when it is needed for the task at hand, instead of being processed (and memorized) in advance (Droll & Hayhoe, 2007;Hayhoe et al, 2003).…”

“…Indeed, the cost of sampling altered the trade-off between storing and just-in-time sampling; shorter distances were linked to the dominance of external sampling, whereas larger distances were associated with more storing (Ballard et al, 1995;Draschkow et al, 2021;Inamdar & Pomplun, 2003). The second set of studies delayed the access to external information, for example by letting participants wait every time they wanted to sample externally (Böing et al, 2023;Gray et al, 2006;Melnik et al, 2018;Sahakian et al, 2023;Somai et al, 2020). There, participants showed similar patterns of behaviour as in the distance manipulations, providing strong evidence that the cost of access to information in the external world shifts the balance of when and how much one relies on VWM.…”