Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction

Abstract

Extracting event temporal relations is a critical task for information extraction and plays an important role in natural language understanding. Prior systems leverage deep learning and pre-trained language models to improve the performance of the task. However, these systems often suffer from two shortcomings: 1) when performing maximum a posteriori (MAP) inference based on neural models, previous systems only used structured knowledge that is assumed to be absolutely correct, i.e., hard constraints; 2) biased predictions on dominant temporal relations when training with a limited amount of data. To address these issues, we propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge. We solve the constrained inference problem via Lagrangian Relaxation and apply it to end-to-end event temporal relation extraction tasks. Experimental results show our framework is able to improve the baseline neural network models with strong statistical significance on two widely used datasets in news and clinical domains.

Temporal Relation Extraction with a Graph-Based Deep Biaffine Attention Model

“…Similar to prior temporal relation extraction systems (Han et al, 2019(Han et al, , 2020, we use a BiLSTM layer to encode the token embeddings from the BERT model. The encoded sentences are then passed into four independent Multilayer Perceptrons (MLP), which we will describe below.…”

“…Structured learning approaches are also proposed to enhance performance by incorporating more knowledge (Ning et al, 2017;Han et al, 2019). Recent research efforts further explore the power of pre-trained Transformer-based models (Vaswani et al, 2017) on temporal relation extraction by proposing ex-tensions and variants (Wang et al, 2019;Yang et al, 2019;Han et al, 2019Han et al, , 2020. Latest state-of-the-art performance has been achieved by end-to-end, multi-task approaches that conduct both event annotation and temporal relations extraction jointly (Han et al, 2019(Han et al, , 2020Lin et al, 2020).…”

“…Recent research efforts further explore the power of pre-trained Transformer-based models (Vaswani et al, 2017) on temporal relation extraction by proposing ex-tensions and variants (Wang et al, 2019;Yang et al, 2019;Han et al, 2019Han et al, , 2020. Latest state-of-the-art performance has been achieved by end-to-end, multi-task approaches that conduct both event annotation and temporal relations extraction jointly (Han et al, 2019(Han et al, , 2020Lin et al, 2020). Nonetheless, even if these systems are able to implement joint information extraction, they suffer from performance issues in the inference speed and accuracy, which may be the result of the computationally expensive decoding process and insufficient fusion of information extracted for classification.…”

OntoED: Low-resource Event Detection with Ontology Embedding

“…Thereinto, R H = {SUBSUPER, SUPERSUB, COSUPER 1 } denotes a set of relation labels defined in the subevent relation extraction task (Wang et al, 2020a;Yao et al, 2020). R T = {BEFORE, AFTER, EQUAL 2 } denotes a set of temporal relations (Han et al, 2020). R C = {CAUSE, CAUSEDBY} denotes a set of causal relations (Ning et al, 2018).…”

EventPlus: A Temporal Event Understanding Pipeline

“…For event extraction, some systems only provide results within a certain defined ontology such as AIDA (Li et al, 2019), there are also some works utilizing data from multiple modalities (Li et al, 2020a,b). Some works could handle novel events (Xiang and Wang, 2019;Ahmad et al, 2021;Han et al, 2020b;, but they are either restricted to a certain domain (Yang et al, 2018) or lack of performance superiority because of their lexico-syntactic rule-based algorithm (Valenzuela-Escárcega et al, 2015). For temporal information detection, proposes a neural-based temporal relation extraction system with knowledge injection.…”