Cognitive Processing of Quantifiers

Szymanik, Jakub

doi:10.1007/978-3-319-28749-2_5

Cited by 2 publications

(2 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relative stability of the core semantics of quantifiers makes a cross-linguistic comparison possible, but the syntactic and pragmatic variation associated with the expression of generalized quantifiers poses a challenge for multilingual NLU. We consult quantifier taxonomy studies (Keenan and Westerståhl, 1997;Peters and Westerståhl, 2006;Szymanik and Thorne, 2015;Szymanik, 2016) and derive a categorization set for quantifier analysis in NLU benchmarks. In coreference (Ogrodniczuk et al, 2019, 2020), negation (Hossain et al, 2020Hartmann et al, 2021), and consistency (Li et al, 2019;Ribeiro et al, 2019;Asai and Hajishirzi, 2020;Geva et al, 2022)-there has been little work on generalized quantifiers as a source of error in NLU, let alone in multilingual NLU.…”

Section: Introductionmentioning

confidence: 99%

Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks

Cui¹,

Hershcovich²,

Søgaard³

2022

Preprint

View full text Add to dashboard Cite

Logical approaches to representing language have developed and evaluated computational models of quantifier words since the 19th century, but today's NLU models still struggle to capture their semantics. We rely on Generalized Quantifier Theory for languageindependent representations of the semantics of quantifier words, to quantify their contribution to the errors of NLU models. We find that quantifiers are pervasive in NLU benchmarks, and their occurrence at test time is associated with performance drops. Multilingual models also exhibit unsatisfying quantifier reasoning abilities, but not necessarily worse for non-English languages. To facilitate directlytargeted probing, we present an adversarial generalized quantifier NLI task (GQNLI) and show that pre-trained language models have a clear lack of robustness in generalized quantifier reasoning.

show abstract

Section: Introductionmentioning

confidence: 99%

Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks

Cui¹,

Hershcovich²,

Søgaard³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The relative stability of the core semantics of quantifiers makes a cross-linguistic comparison possible, but the syntactic and pragmatic variation associated with the expression of generalized quantifiers poses a challenge for multilingual NLU. We consult quantifier taxonomy studies (Keenan and Westerståhl, 1997;Peters and Westerståhl, 2006;Szymanik and Thorne, 2015;Szymanik, 2016) and derive a categorization set for quantifier analysis in NLU benchmarks. In Table 2, we list the 11way quantifier categorization set and their logical denotation based on set theory.…”

Section: Introductionmentioning

confidence: 99%