Semantic memory: A review of methods, models, and current challenges

Abstract

Adult semantic memory has been traditionally conceptualized as a relatively static memory system that consists of knowledge about the world, concepts, and symbols. Considerable work in the past few decades has challenged this static view of semantic memory, and instead proposed a more fluid and flexible system that is sensitive to context, task demands, and perceptual and sensorimotor information from the environment. This paper (1) reviews traditional and modern computational models of semantic memory, within the umbrella of network (free association-based), feature (property generation norms-based), and distributional semantic (natural language corpora-based) models, (2) discusses the contribution of these models to important debates in the literature regarding knowledge representation (localist vs. distributed representations) and learning (error-free/Hebbian learning vs. error-driven/predictive learning), and (3) evaluates how modern computational models (neural network, retrievalbased, and topic models) are revisiting the traditional "static" conceptualization of semantic memory and tackling important challenges in semantic modeling such as addressing temporal, contextual, and attentional influences, as well as incorporating grounding and compositionality into semantic representations. The review also identifies new challenges regarding the abundance and availability of data, the generalization of semantic models to other languages, and the role of social interaction and collaboration in language learning and development. The concluding section advocates the need for integrating representational accounts of semantic memory with process-based accounts of cognitive behavior, as well as the need for explicit comparisons of computational models to human baselines in semantic tasks to adequately assess their psychological plausibility as models of human semantic memory.

A sequential sampling account of semantic relatedness decisions

“…Our findings also tie into an ongoing discussion on disentangling representational (structures) and the cognitive processes that draw on it (Siew, Wulff, Beckage, & Kenett, 2019;Kenett, Beckage, Siew, & Wulff, 2020). It has been argued that computational modeling may be one route to potentially accomplishing this (Kumar, 2020;Wulff et al, 2019). Approaches in this direction typically consider a representational structure, such as a word-vector space or a free-association network, and retrieval processes, such as spreading activation or random walks (Siew et al, 2019).…”

“…The literature has proposed many tasks to behaviorally measure people's representations of semantic relatedness (Kumar, 2020;Wulff et al, 2019). Here we focus on the semantic relatedness decision task (SRDT), a two-alternative, forcedchoice task that requires participants to decide whether two words are semantically related or not.…”

Unveiling the nature of interaction between semantics and phonology in lexical access based on multilayer networks

“…Our linguistic multilayer network is estimated from a unique dataset of free association responses collected in English 28 based on a Big Data approach, collected over several years. www.nature.com/scientificreports/ Such a dataset provides us a unique opportunity to represent a subset of the multidimensional structure of the mental lexicon based on behaviourally collected data 29,30 . Such an approach allows us to empirically examine the relation between phonology and semantics in lexical access, a critical component of linguistic processing, but its nature is still debated.…”

Computational mechanisms for neural representation of words

“…common neighbors) were significant correlated with word2vec models. It remains controversial whether these hyperparameter tuning processes are psychological meaningful and what information is retained and lost after dimension reductions (Kumar, 2021).…”

“…There are significant correlations between word-relational structures constructed from different language computation models, as well as with relational structures derived from the sensory systems (e.g., "cat" and "dog" are closely related across all of these different types of measures) (Kumar, 2021;Lenci, 2018). Understanding the underlying computational architecture (and potentially algorithms) of the language-derived knowledge system requires carefully examining the differences in these models and testing which models better explain the brain activity patterns associated with word meanings.…”