A dynamic neural resource model bridges sensory and working memory

Abstract

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or "iconic" memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these timescales. Here we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures.

Sensory-memory interactions via modular structure explain errors in visual working memory

“…Compared to a previous work adapting efficient coding theories with static tuning curves to account for error patterns in working memory tasks 40 , our extension to memory processes demonstrated how neural activities and behavior readout change dynamically during the delay period. Notably, recent work combined dynamic change of signal amplitude with static tuning curves to capture different time courses of estimation precision during sensory encoding and memory maintenance 41 . Our network models embody such phenomenological models as the networks exhibit changes in overall firing rates after the stimulus offset.…”

Mapping visual working memory models to a theoretical framework

“…Lin & Fougnie, 2022;Myers et al, 2017;Souza & Oberauer, 2016). Tomić and Bays (2023) provide empirical validation that the Dynamic Neural Resource model can accurately model aggregate error distributions across memory arrays with various set sizes and stimulus durations.…”

“…The Neural Resource model was recently updated to incorporate a temporal dimension, accounting for dynamics in the neural activity of sensory areas (iconic memory) that project into WM (the Dynamic Neural Resource model) (Tomić & Bays, 2023). The VWM neuron population accumulates activity from a sensory signal that rapidly decays following stimulus offset.…”

Dissecting the components of error in analogue report tasks

“…Over recent years, studies using variations of this task have shaped our understanding of cognitive functions including visual perception (Bays, 2016;Thibault et al, 2016), attention (Murray et al, 2013;Tang et al, 2020), sensory memory (Pratte, 2018;Tomić & Bays, 2023), working memory (Bays et al, 2009;Fougnie et al, 2012;Schurgin et al, 2020;van den Berg et al, 2012;Zhang & Luck, 2008), and long-term memory (Brady et al, 2013;Richter et al, 2016). Despite its prevalance, a comprehensive study of factors contributing to responses on this task is still lacking.…”

“…While internal noise sets an upper bound on the attainable fidelity, its effects can be amplified by manipulations that decrease signal amplitude. In research on working memory, this can be achieved by increasing the number of presented stimuli, with numerous studies demonstrating a monotonic increase in response error variability with set size (Bays et al, 2009;Ma et al, 2014;Tomić & Bays, 2023;van den Berg et al, 2012). Similarly, studies on sensory and working memory have shown that, in addition to noisy encoding, the maintenance of information in the absence of direct perception introduces additional error in feature reproduction, with the effect becoming increasingly pronounced with retention time (Pratte, 2018;Schneegans & Bays, 2018;Shin et al, 2017).…”