The role of meaning in visual memory: Face-selective brain activity predicts memory for ambiguous face stimuli

Abstract

How people process images is known to affect memory for those images, but these effects have typically been studied using explicit task instructions to vary encoding. Here, we investigate the effects of intrinsic variation in processing on subsequent memory, testing whether recognizing an ambiguous stimulus as meaningful (as a face vs. as shape blobs) predicts subsequent visual memory even when matching the perceptual features and the encoding strategy between subsequently remembered and subsequently forgotten items. We show that single trial EEG activity can predict whether participants will subsequently remember an ambiguous Mooney face image (e.g., an image that will sometimes be seen as a face and sometimes not be seen as a face). In addition, we show that a classifier trained only to discriminate between whether participants perceive a face vs. non-face can generalize to predict whether an ambiguous image is subsequently remembered. Furthermore, when we examine the N170, an ERP index of face processing, we find that images that elicit larger N170s are more likely to be remembered than those that elicit smaller N170s, even when the exact same image elicited larger or smaller N170s across participants. Thus, images processed as meaningful -in this case as a face-during encoding are better remembered than identical images that are not processed as a face. This provides strong evidence that understanding the meaning of a stimulus during encoding plays a critical role in visual memory. Significance StatementIs visual memory inherently visual or does meaning and other conceptual information necessarily play a role even in memory for detailed visual information? Here we show that it's easier to remember an image when it's processed in a meaningful way --as indexed by the amount of category-specific brain activity it elicits. In particular, we use single-trial EEG activity to predict whether an image will be subsequently remembered, and show that the main driver of this prediction ability is whether or not an image is seen as meaningful or non-meaningful. This shows that the extent to which an image is processed as meaningful can be used to predict subsequent memory even when controlling for perceptual factors and encoding strategies that typically differ across images.

Greater visual working memory capacity for visually-matched stimuli when they are recognized as meaningful

“…The observed benefits for working memory performance from recognizing faces and for remembering real-world objects (Brady et al 2016) suggest that associations with meaningful conceptual information is the critical attribute that gives rise to enhanced active storage, similar to the role of meaning in long-term memory (Bower, Karlin, & Dueck, 1975;Konkle et al, 2010;McWeeny, Young, Hay, & Ellis, 1987;Brady et al 2019). What determines the meaningfulness of an object?…”

“…The stimuli used consisted of 300 images (150 ambiguous Mooney faces and 150 shuffled Mooney faces that appeared as unrecognizable shapes). The ambiguous face images were previously ranked as moderately difficult to recognize a face, gaining the status "ambiguous" instead of "easy" (for details, see Schwiedrzik, Melloni, & Schurger, 2018;Brady, Alvarez, & Störmer, 2019). This difficulty was chosen in an attempt to create a more evenly distributed subjective report of faces seen.…”

“…The results of Experiments 1 & 2 provide initial evidence that visual working memory performance may benefit from the perceived meaningfulness of the stimulus. However, changes in behavioral performance alone do not provide conclusive evidence as to whether these stimuli are maintained actively in working memory, or whether episodic memory or activated forms of long-term memory are usedas long-term memory is known to be influenced by such meaning (Bartlett, 1932;Brady et al, 2019;Wiseman & Niesser, 1974;Cowan, 2005). Thus, we next recorded the brain's electrophysiological responses that provide a direct measure of working memory engagement in Experiment 3 (CDA component; Vogel & Machizawa, 2004).…”

“…More details on the experimental design are reported in the Supplementary Material. Electrophysiological Recordings and Analysis EEG activity was recorded continuously using our conventional recording and analysis procedures (Brady, Alvarez , & Störmer, 2019;Keefe & Störmer, 2012). Data were epoched into trials and artifact rejection was performed for the interval -200 to +2000ms relative to the memory display onset and trials with horizontal eye movements, blinks, or excessive muscle movements were excluded from the analysis.…”

“…3). Stimuli consisted of ambiguous face images ("Mooney images"; Mooney, 1957) that were carefully matched so that different images would be recognized as a face by different people (Brady, Alvarez, Störmer, 2019;Schwiedrzik et al 2018); that is, the exact same image could be perceived as meaningful by one person but as not meaningful by the next. Thus, among these face stimuli there were on average no physical differences between images that one participant would perceive as a face or perceive as a set of black-and-white arbitrary shapes (i.e., not meaningful).…”