The ventral visual pathway represents animal appearance over animacy, unlike human behavior and deep neural networks

Bracci, Stefania; Kalfas, Ioannis; Beeck, Hans Op de

doi:10.1101/228932

Cited by 31 publications

(57 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Essentially, this type of models learns to extract visual features that are consistently denoted through a given word; for example on the basis of a large number of examples, the model will learn the visual representation of a table by individuating the typical visual patterns found in the images associated with the label "table". The cognitive plausibility of visual representations obtained from this type of models has been validated in previous studies, which showed that simulations based on estimates from CNNs are in line with explicit intuitions of human participants (Bracci, Kalfas, & de Beeck, 2017;Lazaridou et al, 2017).…”

Section: Image-based Estimates Of Visual Similaritysupporting

confidence: 57%

Data-driven computational models reveal perceptual simulation in word processing

Petilli¹,

Günther²,

Vergallito³

et al. 2019

Preprint

View full text Add to dashboard Cite

In its strongest formulation, grounded cognition claims that “concepts are made up of sensorimotor information”. Following such equivalence, perceptual properties of objects should consistently influence processing even in purely linguistic tasks, where perceptual information is neither solicited nor required. Previous studies tested this prediction in semantic priming tasks, but they did not observe perceptual influence on participants’ performances. However, those findings suffer from critical shortcomings, which may have prevented potential visually grounded/perceptual effects from being detected. Here, we further investigated this topic by applying an innovative method expected to increase the sensitivity in detecting such perceptual effects. Specifically, we adopted an objective, data-driven computational approach to independently quantify vision-based and language-based similarities for prime-target pairs on a continuous scale. We tested whether these measures predicted behavioral performance in a semantic priming mega-study, with various experimental settings. Vision-based similarity was found to facilitate performances, but a dissociation between vision-based and language-based effects was also observed. Thus, in line with grounded cognition, perceptual properties can facilitate word processing in purely linguistic tasks, but the behavioral dissociation challenges extreme claims of sensorimotor and conceptual equivalence.

show abstract

Section: Image-based Estimates Of Visual Similaritysupporting

confidence: 57%

Data-driven computational models reveal perceptual simulation in word processing

Petilli¹,

Günther²,

Vergallito³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition to similarity, other psychological concepts that have been tested in CNNs include typicality [31], Gestalt principles [32], and animacy [33]. In [127], a battery of tests inspired by findings in visual psychophysics were applied to CNNs and CNNs were found to be similar to biological vision according to roughly half of them.…”

Section: Comparison At the Behavioral Levelmentioning

confidence: 99%

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Lindsay

2021

Journal of Cognitive Neuroscience

407

241

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNS in vision research beyond basic object recognition. This is the author's final version. This article has been accepted for publication in the Journal ofCognitive Neuroscience.

show abstract

“…Such models are originally trained to predict an image label from a vector representation encoding the pixel-based RGB values of the respective image (see the upper-right part of Figure 2), and have reached impressive levels of performance in this task (Chatfield, Simonyan, Vedaldi, & Zisserman, 2014;Krizhevsky et al, 2012). Furthermore, representations obtained from such models have been validated as measures of visual similarity (Petilli, Günther, Vergallito, Ciaparelli, & Marelli, 2019), and it has been shown that they closely correspond to human intuitions (Bracci, Ritchie, Kalfas, & de Beeck, 2019;Lazaridou, Marelli, & Baroni, 2017;Phillips et al, 2018;Zhang, Isola, Efros, Shechtman, & Wang, 2018).…”

Section: Vision-based Representationsmentioning

confidence: 99%

Semantic transparency is not invisibility: A computational model of perceptually-grounded conceptual combination in word processing

Günther

Petilli

Marelli

2020

Journal of Memory and Language

View full text Add to dashboard Cite

Note: This is the author's preprint version of the article (date: 28. 01. 2020). The final article is accepted for publication in the Journal of Memory and Language. Previous studies found that an automatic meaning-composition process affects the processing of morphologically complex words, and related this operation to conceptual combination. However, research on embodied cognition demonstrates that concepts are more than just lexical meanings, rather being also grounded in perceptual experience. Therefore, perception-based information should also be involved in mental operations on concepts, such as conceptual combination. Consequently, we should expect to find perceptual effects in the processing of morphologically complex words. In order to investigate this hypothesis, we present the first fullyimplemented and data-driven model of perception-based (more specifically, vision-based) conceptual combination, and use the predictions of such a model to investigate processing times for compound words in four large-scale behavioral experiments employing three paradigms (naming, lexical decision, and timed sensibility judgments). We observe facilitatory effects of vision-based compositionality in all three paradigms, over and above a strong language-based (lexical and semantic) baseline, thus demonstrating for the first time perceptually grounded effects at the sub-lexical level. This suggests that perceptually-grounded information is not only utilized according to specific task demands but rather automatically activated when available.

show abstract

The ventral visual pathway represents animal appearance over animacy, unlike human behavior and deep neural networks

Cited by 31 publications

References 63 publications

Data-driven computational models reveal perceptual simulation in word processing

Data-driven computational models reveal perceptual simulation in word processing

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Semantic transparency is not invisibility: A computational model of perceptually-grounded conceptual combination in word processing

Contact Info

Product

Resources

About