An Improved Neural Baseline for Temporal Relation Extraction

Abstract

Determining temporal relations (e.g., before or after) between events has been a challenging natural language understanding task, partly due to the difficulty to generate large amounts of high-quality training data. Consequently, neural approaches have not been widely used on it, or showed only moderate improvements. This paper proposes a new neural system that achieves about 10% absolute improvement in accuracy over the previous best system (25% error reduction) on two benchmark datasets. The proposed system is trained on the stateof-the-art MATRES dataset and applies contextualized word embeddings, a Siamese encoder of a temporal common sense knowledge base, and global inference via integer linear programming (ILP). We suggest that the new approach could serve as a strong baseline for future research in this area.

Joint Constrained Learning for Event-Event Relation Extraction

“…In Table 2 we report the TempRel extraction results on MATRES. Among the baseline methods, Ning et al (2019) offer the best performance in terms of F 1 by incorporating an LSTM with global inference and commonsense knowledge. In contrast, the proposed joint constrained learning framework surpasses the best baseline method by a relative gain of 3.27% in F 1 , and excels in terms of both precision and recall.…”

“…Recently, data-driven methods have been developed for TempRel extraction, and have offered promising performance. Ning et al (2019) addressed this problem using a system combining an LSTM document encoder and a Siamese multi-layer perceptron (MLP) encoder for temporal commonsense knowledge from TEMPROB (Ning et al, 2018a). Han et al (2019a) proposed a bidirectional LSTM (BiL-STM) with structured prediction to extract Tem-pRels.…”

“…Han et al (2019b) introduce a BiLSTM model that incorporates MAP inference (BiLSTM+MAP). Ning et al (2019) present the SOTA data-driven method incorporating ILP and commonsense knowledge from TEMPROB with LSTM (LSTM+CSE+ILP). We also compare with the CogCompTime system (Ning et al, 2018c).…”

“…MATRES comes with splits of 183, 72 and 20 documents respectively used for training, development and testing. Following the settings in previous work (Ning et al, 2019;Han et al, 2019b), we report the micro-average of precision, recall and F1 scores on test cases. On HiEve, we use the same evaluation setting as Glavaš andŠnajder (2014) and Zhou et al (2020a), leaving 20% of the documents out for testing.…”

“…Such methods often require designing various features to characterize the structural, discourse and narrative aspects of the events, which are costly to produce and are often specific to a certain task or dataset. More recent works attempted to use datadriven methods based on neural relation extraction models (Dligach et al, 2017;Ning et al, 2019;Han et al, 2019a,b) which refrain from feature engineering and offer competent performances.…”