Building semantic memory from embodied and distributional language experience

Davis, Charles P.; Yee, Eiling

doi:10.1002/wcs.1555

Cited by 31 publications

(28 citation statements)

References 125 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…80%) are typically learnt not in real life situations where reference objects and actions are present in the environment of the communicating individuals, but rather from texts (Kintsch, 1974(Kintsch, , 1998. So-called 'hybrid models' of semantics and concepts (Andrews et al, 2009;Davis & Yee, 2021;Glenberg & Robertson, 2000;Harnad, 1990;Louwerse & Jeuniaux, 2010) take into account both relevant facts, that at least some concepts and symbols require conceptual 'grounding' in specific sensorimotor information from the world, that is, in concept-related objects, actions or their features, and that, after such grounding has happened, distributional or other types of learning relating symbols to symbols can function via contextual transfer of conceptual information. The learning of symbolic meaning by way of previously grounded symbols is sometimes called 'indirect grounding', 'grounding transfer' or 'symbolic theft' and is now supported by ample evidence both from behavioural experiments and computational models (e.g.…”

Section: Introductionmentioning

confidence: 99%

Modelling concrete and abstract concepts using brain-constrained deep neural networks

Henningsen-Schomers

Pulvermüller

2021

Psychological Research

View full text Add to dashboard Cite

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically ‘ground’ concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their ‘shared neurons’, thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

show abstract

Section: Introductionmentioning

confidence: 99%

Modelling concrete and abstract concepts using brain-constrained deep neural networks

Henningsen-Schomers

Pulvermüller

2021

Psychological Research

View full text Add to dashboard Cite

show abstract

“…Future research should try to study other potential moderating effects of relevant variables on the relations between amodal maturational processes and emotional propagative processes. Recent reviews of the study of the interaction between the affective and semantic properties of words have brought up the need to regard them as part of a whole system (e.g., Barsalou, Dutriaux, & Scheepers, 2018; Davis & Yee, 2021; Winkielman, Coulson, & Niedenthal, 2018). As stated by Ostarek and Huettig (2019), new paradigms should be endorsed to advance embodiment research.…”

Section: Discussionmentioning

confidence: 99%

“…But the activation of modal representations seems to participate in language processing more than expected in a system formed by purely amodal symbols (e.g., Binder, Westbury, McKiernan, Possing, & Medler, 2005;Hauk, Johnsrude, & Pulvermüller, 2004;Zwaan & Yaxley, 2003;Nastase & Haxby, 2017;Davis & Yee, 2021;Vergallito, Petilli, Cattaneo, & Marelli, 2019). Thus, there is also a need to formalize an embodiment process to deal with the classical claims of the symbol grounding problem (Harnad, 1990;Searle, 1980), as some amodal representations should be grounded to avoid the Chinese Room Argument (Searle, 1980).…”

Section: Introductionmentioning

confidence: 99%

Emotional Valence Precedes Semantic Maturation of Words: A Longitudinal Computational Study of Early Verbal Emotional Anchoring

2021

View full text Add to dashboard Cite

We present a longitudinal computational study on the connection between emotional and amodal word representations from a developmental perspective. In this study, children's and adult word representations were generated using the latent semantic analysis (LSA) vector space model and Word Maturity methodology. Some children's word representations were used to set a mapping function between amodal and emotional word representations with a neural network model using ratings from 9-year-old children. The neural network was trained and validated in the child semantic space. Then, the resulting neural network was tested with adult word representations using ratings from an adult data set. Samples of 1210 and 5315 words were used in the child and the adult semantic spaces, respectively. Results suggested that the emotional valence of words can be predicted from amodal vector representations even at the child stage, and accurate emotional propagation was found in the adult word vector representations. In this way, different propagative processes were observed in the adult semantic space. These findings highlight a potential mechanism for early verbal emotional anchoring. Moreover, different multiple linear regression and mixed-effect models revealed moderation effects for the performance of the longitudinal computational model. First, words with early maturation and subsequent semantic definition promoted emotional propagation. Second, an interaction effect between age of acquisition and abstractness was found to explain model performance. The theoretical and methodological implications are discussed.

show abstract

“…The embodied approach argues that processing conceptual representations requires eliciting sensorimotor experiences, for example, by simulating them (Von Hofsten, 2007). Motor representations are then automatically elicited when we activate conceptual representations (Aziz-Zadeh & Damasio, 2008;Galetzka, 2017;Meteyard et al, 2012), as shown in the context of language (Borghi et al, 2019;Borghi & Binkofski, 2014;Borghi & Cimatti, 2010;Davis & Yee, 2021;Hauk et al, 2004). Note that here the term "sensorimotor experiences" should be intended broadly, to encompass multiple exteroceptive, proprioceptive and interoceptive modalities -all of which can become linked to action representation during the learning of likelihood mappings.…”

Section: Figure 5 Results Of Simulation 3: How the Generative Model I...mentioning

confidence: 99%

An active inference model of hierarchical action understanding, learning and imitation

Proietti¹,

Pezzulo²,

Tessari³

2021

Preprint

View full text Add to dashboard Cite

We advance a novel computational model of the acquisition of a hierarchical action repertoire and its use for observation, understanding and motor control. The model is grounded in a principled framework to understand brain and cognition: active inference. We exemplify the functioning of the model by presenting four simulations of a tennis learner who observes a teacher performing tennis shots and forms hierarchical representations of the observed actions - including both actions that are already in her repertoire and novel actions - and finally imitates them. Our simulations that show that the agent’s oculomotor activity implements an active information sampling strategy that permits inferring the kinematics aspects of the observed movement, which lie at the lowest level of the action hierarchy. In turn, this low-level kinematic inference supports higher-level inferences about deeper aspects of the observed actions, such as their proximal goals and intentions. Finally, the inferred action representations can steer imitative motor responses, but interfere with the execution of different actions. Taken together, our simulations show that the same hierarchical active inference model provides a unified account of action observation, understanding, learning and imitation. Finally, our model provides a computational rationale to explain the neurobiological underpinnings of visuomotor cognition, including the multiple routes for action understanding in the dorsal and ventral streams and mirror mechanisms.

show abstract

Building semantic memory from embodied and distributional language experience

Cited by 31 publications

References 125 publications

Modelling concrete and abstract concepts using brain-constrained deep neural networks

Modelling concrete and abstract concepts using brain-constrained deep neural networks

Emotional Valence Precedes Semantic Maturation of Words: A Longitudinal Computational Study of Early Verbal Emotional Anchoring

An active inference model of hierarchical action understanding, learning and imitation

Contact Info

Product

Resources

About