Gendered Pronoun Resolution using BERT and an Extractive Question Answering Formulation

Chada, Rakesh

doi:10.18653/v1/w19-3819

Cited by 10 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, we excluded them from our counts for techniques as well. We cite the papers here; most propose techniques we would have categorized as "Questionable correlations," with a few as "Other representational harms" (Abzaliev, 2019; Attree, 2019; Bao and Qiao, 2019;Chada, 2019;Ionita et al, 2019;Lois et al, 2019;Wang, 2019;Xu and Yang, 2019;Yang et al, 2019).…”

Section: Acknowledgmentsmentioning

confidence: 99%

Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Blodgett¹,

Barocas²,

Daumé³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

449

223

View full text Add to dashboard Cite

We survey 146 papers analyzing "bias" in NLP systems, fnding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further fnd that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these fndings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"-i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements-and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.

show abstract

Section: Acknowledgmentsmentioning

confidence: 99%

Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Blodgett¹,

Barocas²,

Daumé³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

449

223

View full text Add to dashboard Cite

show abstract

“…Question Answering (QA) is an active area of research in Natural Language Processing and the recent advances in pre-trained language models enabled lots of rapid progress in the field , (Brown et al, 2020), (Bao et al, 2020), (Raffel et al, 2020)). QA is also used as a format to cast several NLP problems (McCann et al, 2018), (Chada, 2019). A common way to build a high performing question answering model is to fine-tune these pre-trained models on the entire training dataset -either via a span-extraction objective (Lan et al, 2020), (Clark et al, 2020b), (Bao et al, 2020) or a span-generation objective (Raffel et al, 2020).…”

Section: Multilingual Resultsmentioning

confidence: 99%

FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models

Chada¹,

Natarajan²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The task of learning from only a few examples (called a few-shot setting) is of key importance and relevance to a real-world setting. For question answering (QA), the current state-of-the-art pre-trained models typically need fine-tuning on tens of thousands of examples to obtain good results. Their performance degrades significantly in a few-shot setting (< 100 examples). To address this, we propose a simple fine-tuning framework that leverages pre-trained text-to-text models and is directly aligned with their pre-training framework. Specifically, we construct the input as a concatenation of the question, a mask token representing the answer span and a context. Given this input, the model is fine-tuned using the same objective as that of its pre-training objective. Through experimental studies on various few-shot configurations, we show that this formulation leads to significant gains on multiple QA benchmarks (an absolute gain of 34.2 F1 points on average when there are only 16 training examples). The gains extend further when used with larger models (Eg:-72.3 F1 on SQuAD using BART-large with only 32 examples) and translate well to a multilingual setting . On the multilingual TydiQA benchmark, our model outperforms the XLM-Roberta-large by an absolute margin of upto 40 F1 points and an average of 33 F1 points in a few-shot setting (<= 64 training examples). We conduct detailed ablation studies to analyze factors contributing to these gains.

show abstract

“…The above evaluation scheme does not use the fact that there are only three marked mentions in each snippet. There are however previous works (Attree, 2019;Chada, 2019) that consider the goldtwo-mention task (Webster et al, 2018), where the locations of the gold names and pronoun are used during inference as well 1 . We will compare our results in both scenarios: detected-mentions, where models need to detect the mentions by themselves, and gold-two-mention.…”

Section: Methodsmentioning

confidence: 99%

Pre-training Mention Representations in Coreference Models

Varkel¹,

Globerson²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Collecting labeled data for coreference resolution is a challenging task, requiring skilled annotators. It is thus desirable to develop coreference resolution models that can make use of unlabeled data. Here we provide such an approach for the powerful class of neural coreference models. These models rely on representations of mentions, and we show these representations can be learned in a self-supervised manner towards improving resolution accuracy. We propose two self-supervised tasks that are closely related to coreference resolution and thus improve mention representation. Applying this approach to the GAP dataset results in new state of the arts results.

show abstract

Gendered Pronoun Resolution using BERT and an Extractive Question Answering Formulation

Cited by 10 publications

References 14 publications

Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Language (Technology) is Power: A Critical Survey of “Bias” in NLP

FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models

Pre-training Mention Representations in Coreference Models

Contact Info

Product

Resources

About