Coreference Resolution through a seq2seq Transition-Based System

Abstract

Most recent coreference resolution systems use search algorithms over possible spans to identify mentions and resolve coreference. We instead present a coreference resolution system that uses a text-to-text (seq2seq) paradigm to predict mentions and links jointly. We implement the coreference system as a transition system and use multilingual T5 as an underlying language model. We obtain state-of-the-art accuracy on the CoNLL-2012 datasets with 83.3 F1-score for English (a 2.3 higher F1-score than previous work [Dobrovolskii, 2021]) using only CoNLL data for training, 68.5 F1-score for Arabic (+4.1 higher than previous work), and 74.3 F1-score for Chinese (+5.3). In addition we use the SemEval-2010 data sets for experiments in the zero-shot setting, a few-shot setting, and supervised setting using all available training data. We obtain substantially higher zero-shot F1-scores for 3 out of 4 languages than previous approaches and significantly exceed previous supervised state-of-the-art results for all five tested languages. We provide the code and models as open source.1

CR‐M‐SpanBERT: Multiple embedding‐based DNN coreference resolution using self‐attention SpanBERT

“…As shown in Table 4, Bohnet and others [26] and Liu and others [25] reported average F1 scores of 83.3 and 82.3 for CR, respectively. However, to achieve such a high performance, they used mT5 XXL (LM with a parameter size of 13 billion) and T0 3B (LM with a parameter size of 3 billion).…”

“…Various language models (LMs), such as LSTM [18], BERT [19], SpanBERT [21], T‐zero (T0) [25], and multilingual pretrained text‐to‐text transfer transformer (mT5) [26], have been utilized to enhance the performance of CR. In CR, mention clusters in NL texts are recognized using the output embeddings of the LM.…”

A Survey on Challenges and Advances in Natural Language Processing with a Focus on Legal Informatics and Low-Resource Languages

“…The current state of the art in the field is presented in [51]. This paper presents a simplified text-to-text (seq2seq) method for Coreference Resolution that synergizes with modern encoder-decoder or decoder-only models.…”

ÚFAL CorPipe at CRAC 2023: Larger Context Improves Multilingual Coreference Resolution

“…The recent work of Bohnet et al (2023) pushes the end-to-end approach even further, solving both mention detection and coreference linking jointly via a text-to-text paradigm, reaching state-of-the-art results on the CoNLL 2012 dataset (Pradhan et al, 2012). Given that our system uses the same pretrained encoder but a custom decoder designed specifically for coreference resolution instead of a general but pretrained decoder, it would be interesting to perform a direct comparison of these systems.…”

“…In the original architecture, we employed largesized models XLM-R large (Conneau et al, 2020) and RemBERT (Chung et al, 2021). However, even bigger models consistently deliver better performance in various applications (Kale and Rastogi, 2020;Xue et al, 2021;Rothe et al, 2021;Bohnet et al, 2023). We therefore decided to utilize the largest possible pretrained multilingual model.…”