Recall of facial expressions and simple orientations reveals competition for resources at multiple levels of the visual hierarchy

Abstract

Many studies of visual working memory have tested humans' ability to reproduce primary visual features of simple objects, such as the orientation of a grating or the hue of a color patch, following a delay. A consistent finding of such studies is that precision of responses declines as the number of items in memory increases. Here we compared visual working memory for primary features and high-level objects. We presented participants with memory arrays consisting of oriented gratings, facial expressions, or a mixture of both. Precision of reproduction for all facial expressions declined steadily as the memory load was increased from one to five faces. For primary features, this decline and the specific distributions of error observed, have been parsimoniously explained in terms of neural population codes. We adapted the population coding model for circular variables to the non-circular and bounded parameter space used for expression estimation. Total population activity was held constant according to the principle of normalization and the intensity of expression was decoded by drawing samples from the Bayesian posterior distribution. The model fit the data well, showing that principles of population coding can be applied to model memory representations at multiple levels of the visual hierarchy. When both gratings and faces had to be remembered, an asymmetry was observed. Increasing the number of faces decreased precision of orientation recall, but increasing the number of gratings did not affect recall of expression, suggesting that memorizing faces involves the automatic encoding of low-level features, in addition to higher-level expression information.

The role of meaning in visual working memory: Real-world objects, but not simple features, benefit from deeper processing.

“…In addition, we have shown that perceptually-matched images that are perceived as a face are not only better remembered in a working memory task than those not perceived as a face, but also elicit a larger CDA (Asp, Störmer & Brady, 2019), once again showing that 'online' storage in working memory nearly always tracks behavioral performance in these tasks, rather than participants relying on a mix of memory systems only for some stimuli at some encoding times but not others. Consistent with this model of greater engagement of higher-level regions with meaningful stimuli, Salmela et al (2019) have shown that storing faces in memory results in the storage of both low-and high-level information about them, whereas simple orientation stimuli are stored in a solely low-level way. Furthermore, a significant literature has shown, using behavior alone, that familiarity and knowledge improve performance in short-term memory tasks even with perceptually well-matched or even identical stimuli (e.g., Alvarez & Cavanagh, 2004;Jackson & Raymond, 2008;Brady et al 2009;Curby et al 2009;Ngiam et al 2019;O'Donnell, Clement, & Brockmole, 2018;Sahar et al 2020;Starr, Srinivasan, Bunge, 2020).…”

Perceived emotional expressions of composite faces

“…Sadness was not included since we expected sadness to be perceived very similarly to the neutral expression, especially when a sad mouth is combined with another expression in the eyes. Our previous study also suggests that sadness is the least precisely identified, discriminated and remembered expression [20].…”

The role of ensemble average differs in working memory for depth and planar information

“… Sims, Jacobs, & Knill, 2012 ; Wilken & Ma, 2004 ) and can be reproduced by Bayesian models that incorporate a prior reflecting the stimulus space (e.g. Salmela, Ölander, Muukkonen, & Bays, 2019 ). The present study did not distinguish whether the contraction bias comes from reporting the average of the tested feature values or a systematic bias to the center of the space, and future investigation is needed to clarify these two accounts.…”