New Insights into Cross-Document Event Coreference: Systematic Comparison and a Simplified Approach

Abstract

Cross-Document Event Coreference (CDEC) is the task of finding coreference relationships between events in separate documents, most commonly assessed using the Event Coreference Bank+ corpus (ECB+). At least two different approaches have been proposed for CDEC on ECB+ that use only event triggers, and at least four have been proposed that use both triggers and entities. Comparing these approaches is complicated by variation in the systems' use of gold vs. computed labels, as well as variation in the document clustering pre-processing step. We present an approach that matches or slightly beats state-of-the-art performance on CDEC over ECB+ with only event trigger annotations, but with a significantly simpler framework and much smaller feature set relative to prior work. This study allows us to directly compare with prior systems and draw conclusions about the effectiveness of various strategies. Additionally, we provide the first cross-validated evaluation on the ECB+ dataset; the first explicit evaluation of the pairwise event coreference classification step; and the first quantification of the effect of document clustering on system performance. The last in particular reveals that while document clustering is a crucial pre-processing step, improvements can at most provide for a 3 point improvement in CDEC performance, though this might be attributable to ease of document clustering on ECB+.

Sequential Cross-Document Coreference Resolution

“…Finally, the contribution of the multi-perspective cosine similarity underscores the importance of cosine similarity as observed by Cremisini and Finlayson (2020). These ablations, including on the importance of document-level information (− CLS) suggest new directions for token and docu- ment representations in coreference.…”

“…Due to the high-quality document clustering 3 , we only observe a drop of ∼1 CoNLL F1 point when using these predicted clusters, compared to the gold document clusters. However, we note that such a small decrease relies on the quality of the clustering, as shown by the larger gap (3 F1 points) observed by Cremisini and Finlayson (2020) with less accurate clusters.…”

“…This is largely due to the more complex nature of event mentions (i.e., a trigger and arguments) and their syntactic diversity (e.g., both verb phrases and noun-phrases). Prior work on event coreference typically involves pairwise scoring between mentions followed by a standard clustering algorithm to predict coreference links (Pandian et al, 2018;Choubey and Huang, 2017;Cremisini and Finlayson, 2020;Meged et al, 2020;Yu et al, 2020b;Cattan et al, 2020), classification over a fixed number of clusters (Kenyon-Dean et al, 2018) and template-based methods (Cybulska and Vossen, 2015b,a). While pairwise scoring (e.g., graphbased models, see §3.7) with clustering is effective, it requires tuned thresholds (for the clustering algorithm) and cannot use already predicted scores to inform later ones, since all scores are predicted independently.…”

Automatic Data Acquisition for Event Coreference Resolution

“…The existing literature on supervised event coreference resolution primarily focuses on designing pairwise classifier based on the surface linguistic features such as lexical features comprising of lemma and part-of-speech tag similarity of event words (Bejan and Harabagiu, 2010;Lee et al, 2012;Liu et al, 2014;Yang et al, 2015;Cremisini and Finlayson, 2020), argument overlap McConky et al, 2012;Sangeetha and Arock, 2012;Bejan and Harabagiu, 2014;Yang et al, 2015;Choubey and Huang, 2017), semantic similarity based on lexical resources such as wordnet (Bejan and Harabagiu, 2010;Liu et al, 2014;Yu et al, 2016) and word embeddings (Yang et al, 2015;Choubey and Huang, 2017;Kenyon-Dean et al, 2018;Barhom et al, 2019;Zuo et al, 2019;Pandian et al, 2020;Sahlani et al, 2020;, and discourse features such as token and sentence distance (Liu et al, 2014;Cybulska and Vossen, 2015). The resulting classifier is used to cluster event mentions.…”

Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference

“…Cross-document coreference resolution of entities and events (CDCR) is an increasingly important problem, as downstream tasks that benefit from coreference annotations -such as question answering, information extraction, and summarization -begin interpreting multiple documents simultaneously. Yet the number of candidate mentions across documents makes evaluating the full 1 Code is available at https://github.com/ Helw150/event_entity_coref_ecb_plus n 2 pairwise comparisons intractable (Cremisini and Finlayson, 2020). For single-document coreference, the search space is pruned with simple recency-based heuristics, but there is no natural corollary to recency with multiple documents.…”