SemEval-2015 Task 4: TimeLine: Cross-Document Event Ordering

Abstract

This paper describes the outcomes of the TimeLine task (Cross-Document Event Ordering), that was organised within the Time and Space track of SemEval-2015. Given a set of documents and a set of target entities, the task consisted of building a timeline for each entity, by detecting, anchoring in time and ordering the events involving that entity. The TimeLine task goes a step further than previous evaluation challenges by requiring participant systems to perform both event coreference and temporal relation extraction across documents. Four teams submitted the output of their systems to the four proposed subtracks for a total of 13 runs, the best of which obtained an F 1 -score of 7.85 in the main track (timeline creation from raw text).

Multi-lingual and Cross-lingual timeline extraction

“…The present work is directly related to the SemEval 2015 task 4, Timeline: Cross-document event ordering (Minard et al, 2015). Its aim is to combine temporal processing and event coreference resolution to extract from a collection of documents a set of timelines of events pertaining to a specific target entity.…”

“…Both approaches were based on applying a pipeline of linguistic processors including Named Entity Recognition, Event and Nominal Coreference Resolution, Named Entity Disambiguation, and temporal processing (Minard et al, 2015). The SPINOZAVU system was further developed in Caselli et al (2015).…”

“…Therefore, those systems focused on event coreference resolution and temporal processing (Minard et al, 2015). Two recent works have been recently published on Track B: an extension of the GPLSIUA system (Navarro-Colorado and Saquete, 2016), and a distant supervision approach using joint inference (Cornegruta and Vlachos, 2016).…”

“…to detect, anchor in time and order the events involving the target entity" (Minard et al, 2015). As we have already mentioned in the previous section, in this work we will focus on Track A (main track), which is the most demanding and realistic setting of the two: systems are given a set of raw text documents and the task is to extract the timelines.…”

“…Recently, SemEval 2015 included a novel task regarding temporal information extraction (Minard et al, 2015). The aim of SemEval 2015 task 4 was to order in a timeline the events in which a target entity is involved.…”