Representation and computation in working memory

Abstract

The ability to sustain internal representations of the sensory environment beyond immediate perception is a fundamental requirement of cognitive processing. In recent years, debates regarding the capacity and fidelity of the working memory (WM) system have driven significant advances in our understanding of the nature of these representations. In particular, there is growing recognition that WM representations are not merely imperfect copies of a perceived object or event, as new experimental tools have revealed that observers possess richer information about the uncertainty in their memories, and take advantage of environmental regularities to use limited memory resources optimally. Meanwhile, computational models of visual WM formulated at different levels of implementation have converged on common principles relating capacity to variability and uncertainty. Here we review recent research in human visual WM from a computational perspective and the latest developments in identifying neural mechanisms that support it.

Mapping visual working memory models to a theoretical framework

“…It may also provide a common language or platform for researchers to discuss their varying perspectives on how the VWM system operates. VWM models have vastly and rapidly evolved since their initial slot or resource conceptions, and the wider field may not have kept track of the key data and subsequent critical changes to the models (Bays et al, 2022). For example, despite the strong object model generally being disproven (Hardman & Cowan, 2015;Olson & Jiang, 2002) and not currently widely believed, its specification and ideas continue to feature in research in various ways (see Robinson et al, 2022;Williams et al, 2022).…”

“…A core tenet of resource models is that working memory representations vary in strength in a continuous manner. I have straightforwardly mapped this on to the binding pool nodes to encapsulate that facets of VWM like variable precision has been predominantly modeled as noise in neural populations (Bays, 2014;Bays et al, 2022;Schneegans et al, 2020;Schneegans & Bays, 2017). Population coding accounts model the representation of VWM as an encoding-decoding process -a variable number of samples generated from a neural population tuned for feature values is read-out to produce a signal that informs response behavior (Bays, 2014;Schneegans et al, 2020).…”

“…Further, key disagreements between current VWM researchers may be more specifically defined by using the above theory map, and then perhaps more meaningfully discussed and contested. Consider the current debate between proponents for an object-based item limit and resource-based accounts of VWM capacity on the existence of 'true guesses' (Adam et al, 2017;Bays et al, 2022;Ngiam et al, 2022;Schurgin et al, 2020) that was reviewed earlier.…”

A dynamic neural resource model bridges sensory and working memory

“…Keeping relevant information in an easily accessible state is vital for adaptive behavior in dynamic environments. In the primate visual system, this requirement is met by visual working memory (VWM), the capacity to actively maintain visual information from milliseconds to seconds after a stimulus disappears from view [1][2][3][4]. While the contents of VWM are frequently updated to reflect changes in the environment and in behavioral priorities, the visual processing hierarchy itself introduces additional layers of dynamism [5,6].…”

Visual working memory phenomena based on categorical tasks replicate using a continuous measure: A simple interpretation and some methodological considerations

“…One is earlier work principally concerned with its role in visual imagery, complex processing and long-term memory (Baddeley & Lieberman, 1980;Logie, 1986;Shepard & Metzler, 1971), although a notable exception to this was the work of Phillips (Phillips, 1974;Phillips & Baddeley, 1971;Phillips & Christie, 1977) that was a direct forerunner of the currently dominant approach to visual working memory. This more recent approach has stemmed from research on visual attention and focuses on a more detailed understanding of the earlier stages of visual memory (e.g., Bays et al, 2009;Kahneman et al, 1992;Luck & Vogel, 1997;Wheeler & Treisman, 2002;Zhang & Luck, 2008; for a review see Bays et al, 2022). Our own work in recent years has attempted to bridge the two fields by applying methods that were originally developed for verbal material to their visual equivalent (for reviews, see Baddeley et al, 2011a, and Hitch et al, 2020.…”