Embracing New Techniques in Deep Learning for Estimating Image Memorability

Needell, Coen D.; Bainbridge, Wilma A.

doi:10.1007/s42113-022-00126-5

Cited by 39 publications

(70 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, while the current study operationalizes memorability as CR, prior work has shown high consistency across individuals in their hit rates and false alarm rates (Bainbridge et al, 2013), as well as eye-movement patterns (Bylinskii et al, 2015). Memorability can also be predicted by deep learning neural networks (Khosla et al, 2015;Needell & Bainbridge, 2021) and results in specific fMRI patterns in visual and mnemonic areas of the brain (Bainbridge et al, 2017;Bainbridge & Rissman, 2018). Future work could explore how these different behavioral and physiological signatures of sustained attention and memorability interact with memory.…”

Section: Dissociable Influences Of Memorability and Attentional Statementioning

confidence: 73%

“…That our memories may in some ways be predetermined suggests tantalizing possibilities for influencing and manipulating what we remember. For example, experimenters could select highly memorable and forgettable images to induce certain memories (Needell & Bainbridge, 2021), or monitor someone's attentional fluctuations over time to identify the best time to present to-be-remembered material (deBettencourt et al, 2018(deBettencourt et al, , 2019. Utilizing both approaches, one could make a particularly unforgettable experience, presenting memorable items when attention is high.…”

Section: Dissociable Influences Of Memorability and Attentional Statementioning

confidence: 99%

See 1 more Smart Citation

Predicting visual memory across images and within individuals

et al. 2022

Self Cite

View full text Add to dashboard Cite

Section: Dissociable Influences Of Memorability and Attentional Statementioning

confidence: 73%

Section: Dissociable Influences Of Memorability and Attentional Statementioning

confidence: 99%

Predicting visual memory across images and within individuals

et al. 2022

Self Cite

View full text Add to dashboard Cite

“…More recently, other studies have demonstrated that both visual and semantic features contribute differentially with regards to both object memory (Hovhannisyan et al, 2021) as well as the typicality-memorability relationship, where visually atypical but semantically prototypical scene images may be the most memorable (Koch et al, 2020). Additionally, recent research in memorability prediction suggests that adding semantic information to a deep neural network improves the prediction of memorability scores (Needell & Bainbridge, 2022). Our results demonstrate a strong semantic primacy in memory which lends additional support to recent findings demonstrating the importance of semantic information in determining what we remember.…”

Section: Discussionmentioning

confidence: 99%

“…These memorability scores can account for upwards of 50% of variance in memory task performance (Bainbridge et al, 2013) and demonstrate remarkable resiliency across tasks and robustness to attention and priming (Bainbridge, 2020). This high consistency allows one to make honed predictions about what people will remember, which could have far-reaching implications for fields including advertising, marketing, public safety (Bainbridge et al, 2019), patient care (Bainbridge, Berron, et al, 2019), and computer vision (Needell & Bainbridge, 2022). However, in spite of these high consistencies in what individuals remember, what specific factors determine the memorability of an image is still largely unknown.…”

Section: Introductionmentioning

confidence: 99%

The Features Underlying the Memorability of Objects

Kramer

Hebart

Baker³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Despite decades of study of memory, it remains unclear what makes an image memorable. There is considerable debate surrounding the underlying determinants of memory, including the roles of semantic (e.g., animacy, utility) and visual features (e.g., brightness) as well as whether the most prototypical or most atypical items are best remembered. Prior studies have relied on constrained stimulus sets, limiting any generalized view of the features that may contribute to memory. Here, we collected over one million memory ratings (N=13,946) for THINGS (Hebart et al., 2019), a naturalistic dataset of 26,107 object images designed to comprehensively sample concrete objects. First, we establish a model of object features that is predictive of image memorability, capturing over half of the explainable variance. For this model, we find that semantic features have a stronger influence than visual features on what people will remember. Second, we examined whether memorability could be accounted for by the typicality of the objects, by comparing human behavioral data, object feature dimensions, and deep neural network features. While prototypical objects tend to be the most memorable, the relationship between memorability and typicality is more complex than a simple positive or negative association and typicality alone cannot account for memorability.SIGNIFICANCE STATEMENTWhy is it that we seem to remember and forget the same things? Our lived experiences differ, but there is remarkable consistency in what is remembered across people. Here, we collected memory performance scores for a comprehensive and diverse collection of natural object images to identify which properties determine our ability to remember. We created a model for predicting memory from object features showing that semantic properties more than visual properties contribute primarily to memorability. Further, we find that it is neither the most prototypical or atypical images that are best remembered, which suggests that typicality alone cannot account for memorability. Our findings challenge decades of prior research that suggest that the most atypical items are most memorable, informing our understanding of the features and organizational principles of memory.

show abstract

“…Instead, memorability may be based on more distributed image features (e.g. a hazy sky, an unusual texture) or even non-localized, high level conceptual features ( Needell & Bainbridge, 2022 ). If salient, low-level features drive eye movements, but not memorability ( Isola et al, 2014 ), this may explain the lack of consistent associations between fixations and memorability.…”

Section: Discussionmentioning

confidence: 99%

The extent and specificity of visual exploration determines the formation of recollected memories in complex scenes

et al. 2022

View full text Add to dashboard Cite

Our visual memories of complex scenes often appear as robust, detailed records of the past. Several studies have demonstrated that active exploration with eye movements improves recognition memory for scenes, but it is unclear whether this improvement is due to stronger feelings of familiarity or more detailed recollection. We related the extent and specificity of fixation patterns at encoding and retrieval to different recognition decisions in an incidental memory paradigm. After incidental encoding of 240 real-world scene photographs, participants ( N = 44) answered a surprise memory test by reporting whether an image was new, remembered (indicating recollection), or just known to be old (indicating familiarity). To assess the specificity of their visual memories, we devised a novel report procedure in which participants selected the scene region that they specifically recollected, that appeared most familiar, or that was particularly new to them. At encoding, when considering the entire scene,subsequently recollected compared to familiar or forgotten scenes showed a larger number of fixations that were more broadly distributed, suggesting that more extensive visual exploration determines stronger and more detailed memories. However, when considering only the memory-relevant image areas, fixations were more dense and more clustered for subsequently recollected compared to subsequently familiar scenes. At retrieval, the extent of visual exploration was more restricted for recollected compared to new or forgotten scenes, with a smaller number of fixations. Importantly, fixation density and clustering was greater in memory-relevant areas for recollected versus familiar or falsely recognized images. Our findings suggest that more extensive visual exploration across the entire scene, with a subset of more focal and dense fixations in specific image areas, leads to increased potential for recollecting specific image aspects.

show abstract

Embracing New Techniques in Deep Learning for Estimating Image Memorability

Cited by 39 publications

References 43 publications

Predicting visual memory across images and within individuals

Predicting visual memory across images and within individuals

The Features Underlying the Memorability of Objects

The extent and specificity of visual exploration determines the formation of recollected memories in complex scenes

Contact Info

Product

Resources

About