Probing Word and Sentence Embeddings for Long-distance Dependencies Effects in French and English

Merlo, Paola

doi:10.18653/v1/w19-4817

Cited by 7 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While some experiments have shown that Recursive Neural Networks can learn the main descriptive properties of long-distance dependencies in English, for example the fact that they obey a uniqueness constraint (only one gap per filler) and also that they obey island constraints (Wilcox et al, 2018), work attempting to replicate finer-grained human judgments for French have failed to show a correlation with human behaviour (Merlo and Ackermann, 2018), while other work on English has found mixed results (Chowdhury and Zamparelli, 2018). Lack of correlation with human grammaticality judgments has also been found in wh-islands and object relative clauses for both French and English (Merlo, 2019). More work will be needed to establish the exact boundaries of quantitative properties in long-distance dependencies across several languages.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Proceedings of the First Workshop on Quantitative Syntax (Quasy, SyntaxFest 2019)

2019

View full text Add to dashboard Cite

The use of corpora for NLP and linguistics has only increased in recent years. In NLP, machine learning systems are by nature data-intensive, and in linguistics there is a renewed interest in the empirical validation of linguistic theory, particularly through corpus evidence. While the first statistical parsers have long been trained on the Penn treebank phrase structures, dependency treebanks, whether natively annotated with dependencies, or converted from phrase structures, have become more and more popular, as evidenced by the success of the Universal Dependency project, currently uniting 120 treebanks in 80 languages, annotated in the same dependency-based scheme. The availability of these resources has boosted empirical quantitative studies in syntax. It has also lead to a growing interest in theoretical questions around syntactic dependency, its history, its foundations, and the analyses of various constructions in dependency-based frameworks. Furthermore, the availability of large, multilingual annotated data sets, such as those provided by the Universal Dependencies project, has made cross-linguistic analysis possible to an extent that could only be dreamt of only a few years ago. In this context it was natural to bring together TLT (Treebanks and Linguistic Theories), the historical conference on treebanks as linguistic resources, Depling (The international conference on Dependency Linguistics), the conference uniting research on models and theories around dependency representations, and UDW (Universal Dependency Workshop), the annual meeting of the UD project itself. Moreover, in order to create a point of contact with the large community working in quantitative linguistics it seemed expedient to create a workshop dedicated to quantitative syntactic measures on treebanks and raw corpora, which gave rise to Quasy, the first workshop on Quantitative Syntax. And this led us to the first SyntaxFest. Because the potential audience and submissions to the four events were likely to have substantial overlap, we decided to have a single reviewing process for the whole SyntaxFest. Authors could choose to submit their paper to one or several of the four events, and in case of acceptance, the program co-chairs would decide which event to assign the accepted paper to.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Future work will have to extend the investigation to other features and to other constructions that have been proposed and discussed in the theory and develop more complex models of intervention similarity. Current work is investigating the morpho-syntactic feature person and models of similarity related to word embeddings (Merlo, 2019).…”

Section: Discussionmentioning

confidence: 99%

Proceedings of the First Workshop on Quantitative Syntax (Quasy, SyntaxFest 2019)

2019

View full text Add to dashboard Cite

show abstract

“…For example, when faced with long-distance dependencies between two feature-sharing items in a sentence (such as those found in questions, relative clauses, pronoun anaphora, and other frequent phenomena), people exhibit effects of interference if there is a third similar element in the sentence (Rizzi, 2004 ; Franck et al, 2015 ). However, this effect of similarity interference is not correlated to the similarity of words calculated statically in a vector space or even dynamically in a neural network model of processing (Merlo and Ackermann, 2018 ; Merlo, 2019 ). The general picture that emerges from all these studies is that word similarity is a rich construct of the human lexicon, and while word embedding spaces represent some fundamental properties of semantic similarity, more nuanced notions, and some grammatically-relevant aspects, may not emerge from such representations.…”

Section: Inspiration From Human Lexical Abilitiesmentioning

confidence: 99%

Beyond the Benchmarks: Toward Human-Like Lexical Representations

Stevenson¹,

Merlo²

2022

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

To process language in a way that is compatible with human expectations in a communicative interaction, we need computational representations of lexical properties that form the basis of human knowledge of words. In this article, we concentrate on word-level semantics. We discuss key concepts and issues that underlie the scientific understanding of the human lexicon: its richly structured semantic representations, their ready and continual adaptability, and their grounding in crosslinguistically valid conceptualization. We assess the state of the art in natural language processing (NLP) in achieving these identified properties, and suggest ways in which the language sciences can inspire new approaches to their computational instantiation.

show abstract

“…While we have not provided in this paper a direct mechanistic model of intervention, the outcome of our quantitative investigations are relevant for the increasing body of computational research that attempts to reverse engineer current neural networks models to establish the boundaries of what they can learn. These studies have concentrated on structural grammatical competence, exemplified by long-distance agreement and relative clauses and islands, phenomena that also trigger locality effects, and have demonstrated that neural networks can learn longdistance dependencies to an interesting extent (Linzen et al 2016;Wilcox et al 2018), but do not fully show intervention effects (Merlo & Ackermann 2018;Merlo 2019). The results of the current paper are relevant for this debate as they demonstrate that any discrepancies between the human results and the machine results are not due to lack of sufficient statistical signal in the data, but are morel likely to be found in properties of the learning algorithms.…”

Section: Extending the Supporting Empirical Evidencementioning

confidence: 99%

Intervention effects in clefts: a study in quantitative computational syntax

Samo

Merlo

2021

Glossa: A Journal of General Linguistics

Self Cite

View full text Add to dashboard Cite

Clefts structures show an important asymmetry in interpretation: subject clefts can provide both corrective or new information foci, while non-subjects (objects, adjuncts) are only corrective. According to Belletti (2015), such an asymmetry arises from the fact that movement deriving subject clefts can target two focus positions, but non-subjects can target only one. In both cases a long-distance dependency is created, triggering locality effects. In this paper, we show that intervention effects causing ungrammaticality in certain configurations give rise to lower-than-expected frequencies in corresponding grammatical configurations. Based on sets of features that play a role in the syntactic computation of locality, we compare the theoretically expected and the actually observed counts of features in a corpus of thirteen syntactically annotated treebanks for three languages (English, French, Italian). We find the quantitative effects predicted by the theory of intervention locality. First, subject clefts, where no intervention is at play, are more frequent than object clefts, where intervention is at play. Secondly, object clefts are less frequent than expected in intervention configuration, while subject clefts are roughly as frequent as expected. Finally, we also find that the differential and direction of difference between expected and observed counts is directly proportional to the number of features that establish the intervention, the strength of the intervention. These results provide a three-fold contribution. First, they extend the empirical evidence in favour of the intervention theory of locality. Second, they provide theory-driven quantitative evidence, thus extending in a novel way the sources of evidence used to adjudicate theories. Finally, the paper provides a blueprint for future theory-driven quantitative investigations.

show abstract

Probing Word and Sentence Embeddings for Long-distance Dependencies Effects in French and English

Cited by 7 publications

References 16 publications

Proceedings of the First Workshop on Quantitative Syntax (Quasy, SyntaxFest 2019)

Proceedings of the First Workshop on Quantitative Syntax (Quasy, SyntaxFest 2019)

Beyond the Benchmarks: Toward Human-Like Lexical Representations

Intervention effects in clefts: a study in quantitative computational syntax

Contact Info

Product

Resources

About