Extracting Event Temporal Relations via Hyperbolic Geometry

Abstract

Detecting events and their evolution through time is a crucial task in natural language understanding. Recent neural approaches to event temporal relation extraction typically map events to embeddings in the Euclidean space and train a classifier to detect temporal relations between event pairs. However, embeddings in the Euclidean space cannot capture richer asymmetric relations such as event temporal relations. We thus propose to embed events into hyperbolic spaces, which are intrinsically oriented at modeling hierarchical structures. We introduce two approaches to encode events and their temporal relations in hyperbolic spaces. One approach leverages hyperbolic embeddings to directly infer event relations through simple geometrical operations. In the second one, we devise an end-to-end architecture composed of hyperbolic neural units tailored for the temporal relation extraction task. Thorough experimental assessments on widely used datasets have shown the benefits of revisiting the tasks on a different geometrical space, resulting in state-of-the-art performance on several standard metrics. Finally, the ablation study and several qualitative analyses highlighted the rich event semantics implicitly encoded into hyperbolic spaces. 1

Improving Event Temporal Relation Classification via Auxiliary Label-Aware Contrastive Learning

“…Compared to ECONET and TGT, which use a larger pre-trained language model, or TGT and HGRU, which use networks with complex structures followed RoBERTa base or BERT Large, TempACL enables a smaller and simpler model which only contains a RoBERTa base and two fully connected layers to achieve the state-of-the-art performance. Method TB-Dense MATRES JCL RoBERTa base -78.8 ECONET RoBERTa Large 66.8 79.3 TGT BERT Large 66.7 80.3 Poincaré Event Embeddings (Tan et al, 2021) RoBERTa base -78.9 HGRU+knowledge (Tan et al, 2021) RoBERTa base -80.5 CERT (Fang et al, 2020) RoBERTa We observe that, TempACL make improvements of 5.37%F 1 and 1.81%F 1 on TB-Dense and MATRES respectively compared with the baseline model. In this section, we first qualitatively analyze key samples, and then we do the ablation experiments to further study the effects of patient strategies and label-aware contrastive learning loss.…”

“…We follow the official split strategy that uses TimeBank and AQUAINT for training and Platinum for testing. We also follow the previous works (Ning et al, 2019;Tan et al, 2021) that randomly select 20 percents of the official train documents as dev set.…”

“…propose the TGT network that integrates both traditional multi-head self-attention and a new temporal-oriented attention mechanism and utilizes a syntactic graph that can explicitly find the connection between two events. Tan et al (2021) propose the Poincaré Event Embeddings method which C C L 2 0 2 2 encodes events into hyperbolic spaces. They argue that the embeddings in the hyperbolic space can capture richer asymmetric temporal relations than the embeddings in the Euclidean space.…”

Temporal Relation Extraction with Joint Semantic and Syntactic Attention

“…Mathur et al [ 43 ] put forward a method leveraging rhetorical discourse features and temporal arguments from semantic role labels as well as local syntactic features through a Gated Relational-GCN model. Tan et al [ 44 ] tried to embed events into hyperbolic spaces and train a classifier to capture temporal relations. Breitfeller et al [ 45 ] developed a novel framework to explore semantic information provided by explicit textual time clues.…”

Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction

“…BEFORE and AFTER that follow the arrows denote the extracted TEMPREL's from the sentences by . proaches, for which recent studies also incorporate advanced learning and inference techniques such as structured prediction (Ning et al, 2017(Ning et al, , 2018bHan et al, 2019;Tan et al, 2021), graph representation (Mathur et al, 2021;Zhang et al, 2022), data augmentation (Ballesteros et al, 2020;Trong et al, 2022), and indirect supervision (Zhao et al, 2021;. These models are prevalently built upon pretrained language models (PLMs) and fine-tuned on a small set of annotated documents, e.g., TimeBank-Dense (Cassidy et al, 2014), MATRES (Ning et al, 2018c), and TDDiscourse (Naik et al, 2019).…”