Annotating Temporal Dependency Graphs via Crowdsourcing

Abstract

We present the construction of a corpus of 500 Wikinews articles annotated with temporal dependency graphs (TDGs) that can be used to train systems to understand temporal relations in text. We argue that temporal dependency graphs, built on previous research on narrative times and temporal anaphora, provide a representation scheme that achieves a good balance between completeness and practicality in temporal annotation. We also provide a crowdsourcing strategy to annotate TDGs, and demonstrate the feasibility of this approach with an evaluation of the quality of the annotation, and the utility of the resulting data set by training a machine learning model on this data set. This data set is publicly available 1 .

DocTime: A Document-level Temporal Dependency Graph Parser

“…(Zhang and Xue, 2018b) provided the the earliest TDT corpus on news data and narrative stories, (Zhang and Xue, 2019) released the first English TDT corpus. Yao et al (2020a) relaxed the assumption of single reference edge in dependency trees to form the improved TDG. (Zhang and Xue, 2018a) built an end-to-end neural temporal dependency parser using BiLSTM and Ross et al (2020b) improved it further incorporating BERT.…”

“…events that are certain to happen vs the ones that might happen), event ambiguity (eg. agreeing to terms of a contract vs signing a contract) and need for complete annotation of all event pairs for precise temporal localization (Yao et al, 2020a).…”

“…Early work densely annotated all pairs of events to address this problem , but was limited to short passages or adjacent sentences due to the n 2 complexity of the task, especially for long documents. Recently this problem formulation was significantly simplified using temporal dependency trees (TDT) (Zhang and Xue, 2019) and temporal dependency graphs (TDG) (Yao et al, 2020a) by only capturing the reference TIMEX or event to build a dependency graph to capture this information. This enabled the development of temporal dependency parsers (Zhang and Xue, 2018a; Ross et al, 2020a) to infer temporal relationships more robustly and efficiently.…”

“…Prior work on temporal relationship extraction and temporal dependency parsing have been mostly limited to news (Zhang and Xue, 2019;Yao et al, 2020a;Pustejovsky et al, 2003a), narrative stories (Zhang and Xue, 2018b; Kolomiyets et al, 2012) or clinical notes (Bethard et al, 2016). In addition to experimenting with existing temporal dependency parsing datasets, we introduce a dataset for temporal dependency graphs in a new domain -contractual documents, where temporal reasoning over events has real world legal and monetary implications for users.…”

UMR-Writer: A Web Application for Annotating Uniform Meaning Representations

“…It also adds a representation for aspect and quantifier scope, which are not part of AMR. At the document level, UMR represents temporal (Zhang and Xue, 2018b,a;Yao et al, 2020) and modal dependencies (Vigus et al, 2019) as well as coreference. UMR abstracts away from syntactic representations and preserves semantic relations within and across sentences.…”