Transformer networks of human conceptual knowledge.

Abstract

We present a computational model capable of simulating aspects of human knowledge for thousands of realworld concepts. Our approach involves a pretrained transformer network that is further fine-tuned on large data sets of participant-generated feature norms. We show that such a model can successfully extrapolate from its training data, and predict human knowledge for new concepts and features. We apply our model to stimuli from 25 previous experiments in semantic cognition research and show that it reproduces many findings on semantic verification, concept typicality, feature distribution, and semantic similarity. We also compare our model against several variants, and by doing so, establish the model properties that are necessary for good prediction. The success of our approach shows how a combination of language data and (laboratory-based) psychological data can be used to build models with rich world knowledge. Such models can be used in the service of new psychological applications, such as the modeling of naturalistic semantic verification and knowledge retrieval, as well as the modeling of real-world categorization, decision-making, and reasoning.

Using Machine Learning to Uncover the Semantics of Concepts: How Well Do Typicality Measures Extracted from a BERT Text Classifier Match Human Judgments of Genre Typicality?

“…1 Despite the impressive performance of models based on BERT (and related language representations) in solving language understanding tasks and reasoning problems, we lack direct evidence that these techniques can be used to produce typicality measures that parallel human judgements. In a tour-de-force analysis, Bhatia and Richie (2022) demonstrated that BERT can reproduce human judgment patterns obtained in a wide variety of previous studies of semantic structures. Generally, these take the form of patterns of agreement/disagreement of "is-a" statements relating subconcepts to concepts, for example, "a penguin is a bird."…”

Predicting Implicit Attitudes with Natural Language Data

“…More recent work has refined DSR predictions with the use of laboratory-based human data. In this work, a small set of human responses are used to train DSR models to predict (out-of-sample) responses for arbitrary words and concepts [8,[43][44][45][46] (see [47] for a review). Like the VEM model, this technique uses a pre-trained language model to specify the underlying representations and associations for thousands of words, but fine-tunes these representations to accurately model a psychological variable of interest.…”

“…Due to the high dimensionality of the word vectors, we used a ridge regression instead of a standard linear regression. Prior work has found that such regression models do a good job at predicting valence ratings for new words [48][49] (also see [43][44][45][46]). Indeed, we verified that the above approach was able to predict valence ratings accurately using a 10-fold cross validation exercise.…”

Inductive reasoning in minds and machines

“…Thus, unsurprisingly, researchers have shown that LLMs are able to capture aspects of human linguistic and semantic processing [40,41,42,43] and even mimic some types of reasoning [33,44]. The Feature-BERT model is one example of this [18]. This model does not only predict the features that people associate with different concepts; it also captures several core patterns of human semantic verification, and by doing so shows how these patterns are the natural byproducts of semantic processing in deep neural networks.…”