WikiCREM: A Large Unsupervised Corpus for Coreference Resolution

Kocijan, Vid; Camburu, Oana-Maria; Creţu, Ana-Maria; Yordanov, Yordan S.; Blunsom, Phil; Lukasiewicz, Thomas

doi:10.18653/v1/d19-1439

Cited by 24 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Much research has focused on resolving under-resourced tasks via semi-supervised learning (Pekar et al, 2014;Kocijan et al, 2019;Hou, 2020) and shared representation based transfer learning (Yang et al, 2017;Cotterell and Duh, 2017;Zhou et al, 2019).…”

Section: Approaches For Under-resourced Tasksmentioning

confidence: 99%

“…Recently, another line of research focused on creating synthetic training data from unlabelled/automatically labelled data using some heuristic patterns. Kocijan et al (2019) used Wikipedia to create WikiCREM, a large pronoun resolution dataset, using heuristic rules based on the occurrence of personal names in sentences. The evaluation of their system on Winograd Schema corpora shows that models pre-trained on the WikiCREM consistently outperform models that do not use it.…”

Section: Approaches For Under-resourced Tasksmentioning

confidence: 99%

See 1 more Smart Citation

Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution

Moosavi

Paun

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Now that the performance of coreference resolvers on the simpler forms of anaphoric reference has greatly improved, more attention is devoted to more complex aspects of anaphora. One limitation of virtually all coreference resolution models is the focus on single-antecedent anaphors. Plural anaphors with multiple antecedents-so-called split-antecedent anaphors (as in John met Mary. They went to the movies)-have not been widely studied, because they are not annotated in ONTONOTES and are relatively infrequent in other corpora. In this paper, we introduce the first model for unrestricted resolution of split-antecedent anaphors. We start with a strong baseline enhanced by BERT embeddings, and show that we can substantially improve its performance by addressing the sparsity issue. To do this, we experiment with auxiliary corpora where split-antecedent anaphors were annotated by the crowd, and with transfer learning models using element-of bridging references and single-antecedent coreference as auxiliary tasks. Evaluation on the gold annotated ARRAU corpus shows that the out best model uses a combination of three auxiliary corpora achieved F1 scores of 70% and 43.6% when evaluated in a lenient and strict setting, respectively, i.e., 11 and 21 percentage points gain when compared with our baseline. 1(1)The Joneses i went to the park. They i had a good time.(2) John and Mary i went to the park. They i had a good time.(3) John i met Mary j in the park. They i,j had a good chat .(4) John likes green i , Mary likes blue j , but Tom likes both colours i,j .

show abstract

Section: Approaches For Under-resourced Tasksmentioning

confidence: 99%

Section: Approaches For Under-resourced Tasksmentioning

confidence: 99%

Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution

Moosavi

Paun

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Liu et al (2019) uses a language model objective to train a memory network, which can resolve coreference links. Kocijan et al (2019) finds pairs of sentences with at least two distinct personal names such that one of them is repeated. One non-first occurrence of the repeated candidate is masked, and the goal is to predict the masked name, given the correct and one incorrect candidates.…”

Section: Fine-tuningmentioning

confidence: 99%

“…Two recent attempts for pre-training coreference models have focused on tasks such as language modeling (Liu et al, 2019) and maskedword-prediction for name resolution (Kocijan et al, 2019). Here we propose self-supervision tasks that train the coreference model directly (rather than just the underlying BERT), resulting in improved mention representations and resolution accuracy.…”

Section: Introductionmentioning

confidence: 99%

Pre-training Mention Representations in Coreference Models

Varkel¹,

Globerson²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Collecting labeled data for coreference resolution is a challenging task, requiring skilled annotators. It is thus desirable to develop coreference resolution models that can make use of unlabeled data. Here we provide such an approach for the powerful class of neural coreference models. These models rely on representations of mentions, and we show these representations can be learned in a self-supervised manner towards improving resolution accuracy. We propose two self-supervised tasks that are closely related to coreference resolution and thus improve mention representation. Applying this approach to the GAP dataset results in new state of the arts results.

show abstract

“…2 The challenge required systems to resolve such ambiguous pronouns to one of two given antecedents. Eight years later, in 2019, a team from Oxford, Imperial, and the Alan Turing Institute (Kocijan et al 2019b) showed that fine-tuning a BERT language model with several million artificial Winograd schema sentences extracted from the Wikipedia could enable that model to correctly resolve over 75% percent of the examples in the Challenge, and also improve the performance of systems in resolving more prosaic examples of personal pronoun anaphors (Kocijan et al 2019a). Now, although this is very impressive and something we should learn from, it is unclear how performance can further improve, given that it reflects general patterns of personal pronoun use and not any specific understanding of coreference.…”

mentioning

confidence: 99%