Word meaning is both categorical and continuous.

Abstract

Most words have multiple meanings, but there are foundationally distinct accounts for this. Categorical theories posit that humans maintain discrete entries for distinct word meanings, as in a dictionary. Continuous ones eschew discrete sense representations, arguing that word meanings are best characterized as trajectories through a continuous state space. Both kinds of approach face empirical challenges. In response, we introduce two novel "hybrid" theories, which reconcile discrete sense representations with a continuous view of word meaning. We then report on two behavioral experiments, pairing them with an analytical approach relying on neural language models to test these competing accounts. The experimental results are best explained by one of the novel hybrid accounts, which posits both distinct sense representations and a continuous meaning space. This hybrid account accommodates both the dynamic, context-dependent nature of word meaning, as well as the behavioral evidence for category-like structure in human lexical knowledge. We further develop and quantify the predictive power of several computational implementations of this hybrid account. These results raise questions for future research on lexical ambiguity, such as why and when discrete sense representations might emerge in the first place. They also connect to more general questions about the role of discrete versus gradient representations in cognitive processes and suggest that at least in this case, the best explanation is one that integrates both factors: Word meaning is both categorical and continuous.

Probing the Representational Structure of Regular Polysemy via Sense Analogy Questions: Insights from Contextual Word Vectors

“…To briefly reiterate some of the points raised in the introduction: we chose BERT for our examination of the representational structure of regular polysemy due to its large size and proven track record in simulating a range of aspects of human cognition (Rogers et al, 2021), as well as its capability to produce contextual word vectors by attending to both the preceding and subsequent contexts of a given target word. However, choosing BERT does not preclude the generalizability of our results to other DSMs, as previous research has reached similar conclusions using a variety of DSMs (Floyd et al, 2021;Lopukhina & Lopukhin, 2016;Trott & Bergen, 2023). More importantly, probing DSMs is a well-established method for examining human language cognition, not only in terms of linguistic behavior, such as response times in lexical decision or naming tasks (Mandera et al, 2017), and eye-tracking-based reading times (Heilbron, van Haren, Hagoort, & de Lange, 2023;Pimentel, Meister, Wilcox, Levy, & Cotterell, 2023), but also in internal cognitive/neural representations aligned with EEG (Ettinger, Feldman, Resnik, & Philips, 2016) and fMRI data (e.g., Schrimpf et al, 2021).…”

“…Although we could envision that some detailed aspects of our findings are shaped by the particular implementational details of the BERT model, we nevertheless predict that our overall insights into regular polysemy should be reasonably robust and generalize to other similar models, as has been observed in other contexts using distributional vectors. This prediction is motivated by the qualitatively similar results that have been obtained using different models when studying other aspects of polysemy (e.g., BERT vs. ELMo in Trott & Bergen, 2023; word2vec vs. Sentence‐BERT in Floyd et al., 2021), as well as the prior success in using a variety of models to successfully study other aspects of regular polysemy (e.g., Lopukhina & Lopukhin, 2016). Of course, this prediction is fundamentally an empirical question which we leave for future work.…”

“…Our work also leverages this splitting assumption based on how it leverages the LR component of the LRcos model to answer sense analogy questions. However, this theory may not be accurate due to the challenges in separating senses, the possibility of a word carrying multiple senses simultaneously, and recent computational and psycholinguistic evidence (Li & Joanisse, 2021; Trott & Bergen, 2023) supporting the graded and continuous representation of specific polysemous senses. For these reasons, we expect that the more nuanced views advocated by this recent work will underlie a full account of regular polysemy.…”