SAM-Net: Integrating Event-Level and Chain-Level Attentions to Predict What Happens Next

Abstract

Scripts represent knowledge of event sequences that can help text understanding. Script event prediction requires to measure the relation between an existing chain and the subsequent event. The dominant approaches either focus on the effects of individual events, or the influence of the chain sequence. However, only considering individual events will lose much semantic relations within the event chain, and only considering the sequence of the chain will introduce much noise. With our observations, both the individual events and the event segments within the chain can facilitate the prediction of the subsequent event. This paper develops self attention mechanism to focus on diverse event segments within the chain and the event chain is represented as a set of event segments. We utilize the event-level attention to model the relations between subsequent events and individual events. Then, we propose the chain-level attention to model the relations between subsequent events and event segments within the chain. Finally, we integrate event-level and chain-level attentions to interact with the chain to predict what happens next. Comprehensive experiment results on the widely used New York Times corpus demonstrate that our model achieves better results than other state-of-the-art baselines by adopting the evaluation of Multi-Choice Narrative Cloze task.

Integrating External Event Knowledge for Script Learning

“…(Wang et al, 2017) utilizes event pair modeling and event order into consideration and achieves better performance. (Lv et al, 2019) proposes to use self attention mechanism to discover event segments as event chain modeling and combine event pair modeling to predict event.…”

“…Following (Granroth-Wilding and Clark, 2016;Wang et al, 2017;Li et al, 2018;Lv et al, 2019), we also adopt the Multi-Choice Narrative Cloze (MCNC) task to evaluate the effectiveness of different models. This task aims to select the right subsequent event from a set of events {e c 1 , e c 2 , • • • , e cm }, where m is the number of candidate event.…”

“…Event chain methods like LSTM (Pichotta and Mooney, 2016) model the event chain to predict the next event. PairLSTM (Wang et al, 2017) and SAM-Net (Lv et al, 2019) both utilize event pair and event chain modeling to predict the subsequent event. SGNN (Li et al, 2018) constructs an event graph and utilizes the graph information.…”

Event Prediction in Big Data Era: A Systematic Survey

“…Therefore, multiple architectures have been proposed to directly process the raw (textual) event descriptions to enable them to be used to predict future event semantics or descriptions. These models share a similar generic framework [96,97,139,195,221,231], which begins by encoding each sequence of words into event representations, utilizing an RNN architecture, as shown in Figure 5. The sequence of events must then be characterized by another higher-level RNN to predict future events.…”

Heterogeneous Graph Neural Networks to Predict What Happen Next

“…Existing work on event representation mainly model event chain from three aspects, the intra-event based (Weber et al, 2018;Granroth-Wilding and Clark, 2016), the individual-event based (Li et al, 2018;Wang et al, 2017) and the event-segment (Lv et al, 2019) based models. These methods concentrate on depicting the relations among homogeneous modeling objectives, e.g., the inter-event relations.…”