Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning

Rambelli, Giulia; Chersoni, Emmanuele; Blache, Philippe; Huang, Chu‐Ren; Lenci, Alessandro

doi:10.18653/v1/w19-3312

Cited by 11 publications

(11 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For starters, the semantics of constructions is unclear, even though their meaning is central to the account. Most examples of constructional meaning are hand-coded, with only very recent, exploratory forays into computational modeling (Perek, 2016 ; Rambelli et al, 2019 ; Busso et al, 2020 ) and linkages to embodied theories of meaning (Bergen and Chang, 2013 ). Providing an account of what constructions at different levels of abstraction mean, and how that meaning can be acquired through linguistic experience, is a crucial step for making this program viable and coherent with the assumptions of usage-based approaches.…”

Section: Discussionmentioning

confidence: 99%

Models of Language and Multiword Expressions

Kallens

Christiansen

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

Traditional accounts of language postulate two basic components: words stored in a lexicon, and rules that govern how they can be combined into meaningful sentences, a grammar. But, although this words-and-rules framework has proven itself to be useful in natural language processing and cognitive science, it has also shown important shortcomings when faced with actual language use. In this article, we review evidence from language acquisition, sentence processing, and computational modeling that shows how multiword expressions such as idioms, collocations, and other meaningful and common units that comprise more than one word play a key role in the organization of our linguistic knowledge. Importantly, multiword expressions straddle the line between lexicon and grammar, calling into question how useful this distinction is as a foundation for our understanding of language. Nonetheless, finding a replacement for the foundational role the words-and-rules approach has played in our theories is not straightforward. Thus, the second part of our article reviews and synthesizes the diverse approaches that have attempted to account for the central role of multiword expressions in language representation, acquisition, and processing.

show abstract

Section: Discussionmentioning

confidence: 99%

Models of Language and Multiword Expressions

Kallens

Christiansen

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…We follow the approach for calculating prototypical argument representations by Lenci (2011) and compute a prototype representation for the event patient slot as the centroid vector representations from the most associated entities with the predicate and agent in the sentence. However, instead of computing updates to the prototype using Distributional Memory vectors (as in Lenci, 2011), we here do the same computations using FastText (Bojanowski, Grave, Joulin, & Mikolov, 2017) static embeddings (see also Rambelli, Chersoni, Lenci, Blache, & Huang, 2020). A sentence's plausibility score is computed as the cosine similarity between the FastText embedding of the proposed patient and the relevant prototype vector.…”

Section: Methodsmentioning

confidence: 99%

Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely

Kauf,

Ivanova,

Rambelli

et al. 2023

Cognitive Science

View full text Add to dashboard Cite

Word co‐occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs’ semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pretrained LLMs (from 2018's BERT to 2023's MPT) assign a higher likelihood to plausible descriptions of agent−patient interactions than to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n = 1215), we found that pretrained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign a higher likelihood to possible versus impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely versus unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow‐up analyses, we show that (i) LLM scores are driven by both plausibility and surface‐level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.

show abstract

“…The phenomena of coercion and metonymic interpretation have widely been investigated in NLP, either with classical distributional models (Zarcone and Padó, 2011;Zarcone et al, 2012;Chersoni et al, 2017;McGregor et al, 2017;Chersoni et al, 2021b) or with Transformer-based language models (Rambelli et al, 2020;Pedinotti and Lenci, 2020;Gu, 2022). Most studies focused on complement coercion, a type clash between an event selecting verb and an entity denoting noun, that triggers a hidden event interpretation (e.g.…”

Section: Modeling Conceptual Shifts In Computational Semanticsmentioning

confidence: 99%

Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP

2023

View full text Add to dashboard Cite

As researchers achieve unprecedented technological breakthroughs in natural language processing, the need to understand the systems underlying these advances is more pertinent than ever. BlackboxNLP, now in its sixth iteration, has played an important role in bringing together scholars from a diverse range of backgrounds in order to rigorously study the behavior, representations, and computations of "black-box" neural network models. Our workshop showcases original, cutting-edge research on topics including but not limited to:• analysis of representations via probing and related techniques;• explanation methods such as feature attribution, free-text explanations, or structured explanations; • interdisciplinary methods (e.g., from neuroscience, cognitive science, computer vision, etc.); • interpretable architectures and neural network modules;• mechanistic interpretability and reverse engineering of neural computations; • open-source tools for analysis, visualization, and/or explanation; • opinions about the state of interpretability and explainable NLP; and • targeted evaluations using simplified or formal languages.The sixth BlackboxNLP workshop will be held in Singapore on December 7, 2023, hosted by the Conference on Empirical Methods in Natural Language Processing (EMNLP). 29 full papers and 17 non-archival extended abstracts were accepted for in-person and online presentations, from a total of 66 submissions. This year's workshop will also feature 19 papers on interpretability from the Findings of the ACL: EMNLP 2023, as well as two invited talks and a panel discussion with experts in the field.BlackboxNLP 2023 would not have been possible without the high-quality peer reviews submitted by our program committee, as well as the logistical assistance provided by the EMNLP organizing committee. We gratefully acknowledge financial support from our sponsors, Google and Apple. Our invited speakers, panelists, authors, and presenters have allowed us to put together an outstanding program for all participants to enjoy.Welcome to BlackboxNLP! We look forward to seeing you in Singapore and online.

show abstract

Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning

Cited by 11 publications

References 47 publications

Models of Language and Multiword Expressions

Models of Language and Multiword Expressions

Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely

Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP

Contact Info

Product

Resources

About