Event Time Extraction with a Decision Tree of Neural Classifiers

Abstract

Extracting the information from text when an event happened is challenging. Documents do not only report on current events, but also on past events as well as on future events. Often, the relevant time information for an event is scattered across the document. In this paper we present a novel method to automatically anchor events in time. To our knowledge it is the first approach that takes temporal information from the complete document into account. We created a decision tree that applies neural network based classifiers at its nodes. We use this tree to incrementally infer, in a stepwise manner, at which time frame an event happened. We evaluate the approach on the TimeBank-EventTime Corpus (Reimers et al., 2016) achieving an accuracy of 42.0% compared to an inter-annotator agreement (IAA) of 56.7%. For events that span over a single day we observe an accuracy improvement of 33.1 points compared to the state-of-the-art CAEVO system (Chambers et al., 2014). Without retraining, we apply this model to the SemEval-2015 Task 4 on automatic timeline generation and achieve an improvement of 4.01 points F1-score compared to the state-of-the-art. Our code is publically available.

Annotating Temporal Information in Clinical Notes for Timeline Reconstruction: Towards the Definition of Calendar Expressions

“…In other studies, it has been proposed to add timeline information directly as event attributes, e.g. Reimers et al (2018).…”

“…anchoring events in absolute time. An approach that aimed to anchor events in time and simplify the annotation task was proposed by Reimers et al (2016Reimers et al ( , 2018, where the event time is modelled as an argument of the event mention. However, the emphasis lies on the events, not on time expressions.…”

TDDiscourse: A Dataset for Discourse-Level Temporal Ordering of Events

“…TimeBank and the TempEval tasks (Verhagen et al, 2007(Verhagen et al, , 2010UzZaman et al, 2013) spurred the development of many temporal ordering systems (UzZaman and Allen, 2010;Llorens et al, 2010;Strötgen and Gertz, 2010;Chang and Manning, 2012;Chambers, 2013;Bethard, 2013). More recently, TimeBank-Dense and EventTime prompted development of newer models Mirza and Tonelli, 2016;Cheng and Miyao, 2017;Reimers et al, 2018). Most systems built for TimeBank/ TimeBank-Dense focus on TLINKs between events in the same or adjacent sentences, relying on local features rather than document-level structure, with some exceptions.…”

“…Bramsen et al (2006); Do et al (2012) also incorporate document-level structure, but focus on different corpora. Reimers et al (2018) develop a model for EventTime, which uses a decision tree of CNNs to associate each event from a document with a time. Several works have explored techniques to incorporate document-level cues such as event coreference (Do et al, 2012;Llorens et al, 2015) and causality (Do et al, 2012;Ning et al, 2018a) in temporal ordering systems.…”

Temporal Information Extraction by Predicting Relative Time-lines

“…Because prediction of relative time-lines trained on TimeML-style annotations is new, we cannot compare our model directly to relation extraction or classification models, as the latter do not provide completely temporally consistent TLinks for all possible entity pairs, like the relative timelines do. Neither can we compare directly to existing absolute time-line prediction models such as Reimers et al (2018) because they are trained on different data with a very different annotation scheme.…”