Geometry of visual working memory information in human gaze patterns

Abstract

In decoding visual working memory information from neural signals, stimulus-dependent eye movements have been considered a confound. Here, we combined eye-tracking with representational geometry analyses to uncover this very information in miniature gaze patterns while participants (n = 41) were cued to maintain visual object orientations. Although participants were discouraged from breaking fixation via real-time feedback, small gaze shifts (< 1 degree) robustly encoded the to-be-maintained stimulus orientation, with evidence for encoding two sequentially presented orientations at the same time. While the orientation encoding upon stimulus presentation was object-specific, it changed to a more object-independent format during cued maintenance, particularly when attention had been temporarily withdrawn from the memorandum. Finally, categorical reporting biases increased after unattended storage, with indications of biased gaze geometries emerging already during the maintenance periods prior to behavioral reporting. These findings disclose a wealth of information in gaze patterns during visuospatial working memory, and suggest systematic changes in representational format when memory contents have been unattended.

Microsaccades track location-based object rehearsal in visual working memory

“…By focusing on object rehearsal through location, our findings are also distinct from complementary recent studies that have linked directional biases in microsaccades to the rehearsal of visual shape and orientation information (Dotson et al, 2018; Linde-Domingo & Spitzer, 2022; Mostert et al, 2018; Quax et al, 2019; Thielen et al, 2019). In contrast to these studies where microsaccades directly tracked the target-relevant memory feature (shape or orientation), we used microsaccades to track incidental object-locations that were never queried for report.…”

“…They show how microsaccades cannot be neglected in neuroscience studies of visual working memory. Several recent reports already made clear how microsaccades may confound neural decoding of memory representations, by showing how microsaccades can be systematically biased by task-relevant object-features such as shape and orientation (Dotson et al, 2018; Linde-Domingo & Spitzer, 2022; Mostert et al, 2018; Quax et al, 2019; Thielen et al, 2019). Building on this work, our data make clear how systematic biases in microsaccades can be a concern also for arbitrarily chosen spatial stimulus-configurations at encoding, even if stimulus configurations are never explicitly relevant for the participant.…”

Microsaccades Track Location-Based Object Rehearsal in Visual Working Memory

“…By focusing on object rehearsal through location, our findings are also distinct from complementary recent studies that have linked directional biases in microsaccades to the rehearsal of visual shape and orientation information (Dotson et al, 2018;Mostert et al, 2018;Quax et al, 2019;Thielen et al, 2019;Linde-Domingo and Spitzer, 2022). In contrast to these studies where microsaccades directly tracked the targetrelevant memory feature (shape or orientation), we used microsaccades to track incidental object-locations that were never queried for report.…”

“…They show how microsaccades cannot be neglected in neuroscience studies of visual working memory (see also : Liu et al, 2022: Liu et al, , 2023b. Several recent reports already made clear how microsaccades may confound neural decoding of memory representations, by showing how microsaccades can be systematically biased by task-relevant object-features such as shape and orientation (Dotson et al, 2018;Mostert et al, 2018;Quax et al, 2019;Thielen et al, 2019;Linde-Domingo and Spitzer, 2022). Building on this work, our data make clear how systematic biases in microsaccades can be a concern also for arbitrarily chosen spatial stimulus-configurations at encoding, even if stimulus configurations are never explicitly relevant for the participant.…”

A gradual transition toward categorical representations along the visual hierarchy during working memory, but not perception

“…This conclusion was drawn from the ability to cross generalize from sensory evoked responses, to responses recorded during the delay of a working memory task, using multivariate decoding techniques. However, there are multiple clues that VWM representations can be abstracted away from sensory evoked responses (Kwak & Curtis, 2022;Favila, Kuhl & Winawer, 2022;Rademaker et al, 2019;Linde-Domingo & Spitzer, 2022;Yan et al, 2023), including the considerable differences between perceptual and working memory geometries unveiled in the present analyses. From a conceptual point of view there may also be good reasons to keep formats distinct, as having identical representations for visual inputs and visual memories might make it difficult to distinguish external reality from internally generated thought (Bettencourt & Xu, 2016;Xu 2017;2018).…”