CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures

Abstract

Learning commonsense causal and temporal relation between events is one of the major steps towards deeper language understanding. This is even more crucial for understanding stories and script learning. A prerequisite for learning scripts is a semantic framework which enables capturing rich event structures. In this paper we introduce a novel semantic annotation framework, called Causal and Temporal Relation Scheme (CaTeRS), which is unique in simultaneously capturing a comprehensive set of temporal and causal relations between events. By annotating a total of 1,600 sentences in the context of 320 five-sentence short stories sampled from ROCStories corpus, we demonstrate that these stories are indeed full of causal and temporal relations. Furthermore, we show that the CaTeRS annotation scheme enables high inter-annotator agreement for broad-coverage event entity annotation and moderate agreement on semantic link annotation. 1 These stories can be found here: http://cs. rochester.edu/nlp/rocstories 2 Each of these stories have the following major characteristics: is realistic, has a clear beginning and ending where something happens in between, does not include anything irrelevant

Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation

“…These CAUSES and PRECONDITION labels have been noted to generally combine with temporal information, and therefore annotators annotate causality with one of four fused labels: BEFORE/CAUSES, OVERLAP/CAUSES, BEFORE/PRECONDITION, and OVERLAP/PRECONDITION. This distinction has similarly been suggested in (Mostafazadeh et al, 2016), and bears practical similarity to the decisions in Hong et al (2016) to allow multiple labels between two events, or the layered annotation of Mirza et al (2014) on top of temporal structure.…”

“…Corpora for event-event and event-time relations have also been developed, both for temporal information in the TimeML tradition (Pustejovsky et al, 2003;Minard et al, 2016), and causal structure (Bethard, 2007;Mostafazadeh et al, 2016;Mirza et al, 2014;Hong et al, 2016;Dunietz et al, 2015). Subevent relations corpora have also been annotated (Glava and najder, 2014;Hong et al, 2016).…”

“…Causation has often been divided into CAUSE, EN-ABLE and PREVENT, as outlined in Hobbs (2005) and Wolff (2007), and implemented in Mirza et al (2014) and Mostafazadeh et al (2016). RED annotation, based on preliminary studies of causal annotation in (Ikuta et al, 2014), adopted a twoway distinction between CAUSES and PRECONDI-TION similar to the distinction often made between "Cause" and "enable".…”

Proceedings of the Events and Stories in the News Workshop

“…The corpus has been developed by applying annotation guidelines designed to mark-up the network of explanatory relations which can be realized between pairs of events in a document belonging to a specific topic. Furthermore, the guidelines are compliant with other initiatives for event annotation: temporal processing (TimeML (Pustejovsky et al, 2003a) and Richer Event Description (RED) ), event coreference (Event Coreference Bank+ (ECB+) (Cybulska and Vossen, 2014b)), and causal relations (Causal-TimeBank (Mirza and Tonelli, 2016), BECauSE (Dunietz et al, 2015), ROCStories (Mostafazadeh et al, 2016b) among others).…”

The State of the Art in Semantic Representation

“…See (Mirza et al, 2014) and (Dunietz et al, 2015) for recently proposed annotation schemes for causality and its sub-types. Mostafazadeh et al (2016) integrated causal and TimeML-style temporal relations into a unified representation.…”