Constructing a cross-document event coreference corpus for Dutch

Abstract

Event coreference resolution is a task in which different text fragments that refer to the same real-world event are automatically linked together. This task can be performed not only within a single document but also across different documents and can serve as a basis for many useful Natural Language Processing applications. Resources for this type of research, however, are extremely limited. We compiled the first large-scale dataset for cross-document event coreference resolution in Dutch, comparable in size to the most widely used English event coreference corpora. As data for event coreference is notoriously sparse, we took additional steps to maximize the number of coreference links in our corpus. Due to the complex nature of event coreference resolution, many algorithms consist of pipeline architectures which rely on a series of upstream tasks such as event detection, event argument identification and argument coreference. We tackle the task of event argument coreference to both illustrate the potential of our compiled corpus and to lay the groundwork for a Dutch event coreference resolution system in the future. Results show that existing NLP algorithms can be easily retrofitted to contribute to the subtasks of an event coreference resolution pipeline system.

A Benchmark for Dutch End-to-End Cross-Document Event Coreference Resolution

“…From these studies, it became clear that outward lexical similarity was the prime indicator of coreference, with features based on (partial) string matching being most effective [7]. Additionally, features modeling document structure had some limited success in withindocument contexts [7,25]. More recently, however, feature-based approaches in coreference resolution have been challenged by transformer-based approaches in which large language models are used to generate strong contextual mention representations, which are then used as a basis for the classification algorithms [26,27].…”

The Fourth Workshop on Insights from Negative Results in NLP

“…Interestingly, earlier work on featurebased classifiers for ECR has shown that discourse and meta-linguistic information surrounding an event are in fact important, to some degree, for the classification of coreference (Lu and Ng, 2018). In this paper, we will devise a series of linguistic probes in order to gauge a Dutch transformer-based coreference model's understanding of certain discourse and meta-linguistic event traits that have been shown to be important for within-document ECR (De Langhe et al, 2022c;Lu and Ng, 2018). Currently, it is assumed that this type of information is implicitly encoded into the transformer's contextual embeddings, but with this paper we intend to verify this.…”

“…Our data consists of the Dutch ENCORE corpus (De Langhe et al, 2022a), which includes 15,407 events spread over 1,015 documents that were sourced from a Dutch newspaper article collection (Vermeulen, 2018). The corpus is comparable in size to most large-scale English-language ECR datasets.…”

“…It includes event coreference annotation on both the within-and cross-document level and meta-linguistic information such as the event's prominence (is it a main event or does it provide background information), realis (does the event happen with certainty) and implicit sentiment (positive/negative/neutral). For our probing experiments, we adhere to an identical split of the data as in the original model paper (De Langhe et al, 2022c). We reserve 85% of data for fine-tuning (70% for training and 15% for development) and use the remaining 15% of data for our probing experiments.…”

“…The ECR model consists of the fine-tuned Dutch BERT model BERTje (de Vries et al, 2019). While this BERTje model has been outperformed by Dutch RobBERTa-based models on most standard NLP tasks (Delobelle et al, 2020(Delobelle et al, , 2022, it is still the model of choice for discourse-type tasks such as coreference resolution, which often require the encoding of long-range semantic and syntactic information (De Langhe et al, 2022c).…”