Conditional Generation of Temporally-ordered Event Sequences

Abstract

Models of narrative schema knowledge have proven useful for a range of event-related tasks, but they typically do not capture the temporal relationships between events. We propose a single model that addresses both temporal ordering, sorting given events into the order they occurred, and event infilling, predicting new events which fit into an existing temporally-ordered sequence. We use a BARTbased conditional generation model that can capture both temporality and common event co-occurrence, meaning it can be flexibly applied to different tasks in this space. Our model is trained as a denoising autoencoder: we take temporally-ordered event sequences, shuffle them, delete some events, and then attempt to recover the original event sequence. This task teaches the model to make inferences given incomplete knowledge about the events in an underlying scenario. On the temporal ordering task, we show that our model is able to unscramble event sequences from existing datasets without access to explicitly labeled temporal training data, outperforming both a BERT-based pairwise model and a BERT-based pointer network. On event infilling, human evaluation shows that our model is able to generate events that fit better temporally into the input events when compared to GPT-2 story completion models.

Salience-Aware Event Chain Modeling for Narrative Understanding

“…Our case study on salienceaware discourse parsing shows the advantage of combining event-level and sentence-level salience information. We plan to use these event chain patterns on other narrative understanding and generation tasks, such as constrained story generation (Peng et al, 2018), event script generation Lyu et al, 2021), and implicit event prediction (Lin et al, 2021;Zhou et al, 2021).…”

Coherent Story Generation with Structured Knowledge

“…Then, story Y is completed according to plot P . We use a phrase containing a predicate to represent an event in a sentence because giving an informative representation helps models capture dependencies in the context (Lin et al, 2021). We apply dependency parsing to recognize the root and its object and retain all the words between them.…”

Mitigating Reporting Bias in Semi-supervised Temporal Commonsense Inference with Probabilistic Soft Logic

“…Recently, significant works have been done on the temporal common sense (TCS) reasoning. These works include but are not limited to event duration prediction (Pan, Mulkar, and Hobbs 2006;Vashishtha, Van Durme, and White 2019), scripting learning (i.e., what happens next after the certain events) (Li, Ding, and Liu 2018), event infilling (i.e., predict the implicit event in a temporally-ordered event sequence) (Lin, Chambers, and Durrett 2021;Zhou et al 2021a) and various temporal reasoning based question answering tasks (Zhou et al 2019;Qin et al 2021). As human annotation on the TCS is costly, a surge of works harnesses cheap supervision methods to collect large amount of TCS data and learn reasoning model upon it.…”

“…As human annotation on the TCS is costly, a surge of works harnesses cheap supervision methods to collect large amount of TCS data and learn reasoning model upon it. For example, (Lin, Chambers, and Durrett 2021) utilizes narrative documents corpus to automatically construct data for temporal ordering and event infilling task. (Zhou et al 2020) jointly models three key dimensions of TCS (duration, frequency, and typical time) and the other two auxiliary dimension of TCS, the data of which is also mined from unannotated free text.…”

“…Many natural language processing problems, including information retrieval (Ning et al 2018;Vashishtha, Van Durme, and White 2019), summarization (Yan et al 2011), causal inference (Noah Weber 2020) and reading comprehension (Ning et al 2020;), rely on system's time understanding ability to give correct answers or predictions. Hence, recent NLP systems dedicate to incorporate temporal common sense (TCS) knowledge (e.g., duration and frequency of events) to improve time understanding ability Lin, Chambers, and Durrett 2021). While manually annotating large-scale TCS knowledge is time and labor consuming.…”