Role of time in binding features in visual working memory.

Abstract

Previous research on feature binding in visual working memory has supported a privileged role for location in binding an object's non-spatial features. However, humans are able to correctly recall feature conjunctions of objects that occupy the same location at different times. In a series of behavioral experiments, we investigated binding errors under these conditions, and specifically tested whether ordinal position can take the role of location in mediating feature binding. We performed two dual-report experiments in which participants had to memorize three colored shapes presented sequentially at the screen center. When participants were cued with the ordinal position of one item and had to report its shape and color, report errors for the two features were largely uncorrelated. In contrast, when participants were cued e.g. with an item's shape and reported an incorrect ordinal position, they had a high chance of making a corresponding error in the color report. This pattern of error correlations closely matched the predictions of a model in which color and shape are bound to each other only indirectly via an item's ordinal position. In a third experiment, we directly compared the roles of location and sequential position in feature binding. Participants viewed a sequence of colored disks displayed at different locations, and were cued either by a disk's location or its ordinal position to report its remaining properties. The pattern of errors supported a mixed strategy with individual variation, suggesting that binding via either time or space could be used for this task.

Temporal and spatial reference frames in visual working memory are defined by ordinal and relational properties.

“…It is entirely conceivable that spatiotemporal context does not only support retrieval but also earlier stages of visual working memory processing, for example the maintenance stage by facilitating processes such as individuation or attentional refreshing. The specific function fulfilled by the representation of memory contents within their spatiotemporal context might even take slightly different forms, depending on the respective task demands (e.g., establishing correspondence for change detection or ensuring reliable access for report), and become manifest in different behavioral signatures (e.g., an increase in false alarm rate as in the present study, or specific patterns of error correlations as in Schneegans et al, 2022).…”

“…Regardless of the relative extent of memory impairments associated with spatial or temporal transformations, our results show that spatial and temporal reference frames share the same metrical properties, which dovetails nicely with other findings indicating a functional equivalence of space and time in visual working memory. For instance, both spatial location and temporal (ordinal) position mediate the binding of surface features like color or shape (Schneegans et al, 2022), memory items that are spatially or temporally close are more easily confused (e.g., Rerko et al, 2014; Sapkota et al, 2016; Schneegans et al, 2021), prioritization of visual working memory contents based on temporal position is as direct, fast and effective as prioritization based on spatial location (Heuer & Rolfs, 2022) and the removal of distinctive but task-irrelevant spatial or temporal properties at retrieval interferes with memory, whereas no similar costs were observed when distinctive variations in other task-irrelevant feature dimensions were taken away (Heuer & Rolfs, 2021). Moreover, memory appears to rely more on either space or time, depending on the distribution of items in either domain and hence its usefulness for item individuation: When items are spatially close, temporal separation can be leveraged to differentiate between items, and vice versa (Heuer & Rolfs, 2021; see also Schneegans et al, 2021).…”

“…For one, they might reduce uncertainty and thus contribute to ensuring reliable access to memoranda. If visual working memory contents are (automatically) bound to their relative spatial and temporal positions (Heuer & Rolfs, 2021; Schneegans et al, 2022), their retrieval may involve the activation of their position along either context dimension. As content and context dimensions, as well as their bindings, are represented with limited precision, retrieval is subject to uncertainty (Oberauer & Lin, 2017).…”

The effect of memory load on object reconstruction: Insights from an online mouse-tracking task

“…Early influential models considered binding to operate over independent feature maps, with attention serving as the "glue" between features sharing common spatial locations (Treisman & Gelade, 1980). This seminal work led to debates about the format of the object representation, with experiments positing shape and color features can be bound directly to each other (Luck & Vogel, 1997;Zhang & Luck, 2008), bound by virtue of space or time (Schneegans & Bays, 2017;Schneegans, McMaster, & Bays, 2022), or bound hierarchically as both integrated wholes and independent features (Hedayti, O'Donnell, & Wyble, 2022;A. Y. Li, OBJECT RECONSTRUCTION 6 Fukuda, & Barense, 2022).…”

Neural Distributed Representations for Artificial Intelligence and Modeling of Thinking

“…Experiments in [27] made it possible to propose an episode recall model in which events can "scroll" forward until the beginning of the next event. In [28], it was experimentally shown that the representation of an object can be bound with both time and position.…”