Picking Apart Story Salads

Abstract

During natural disasters and conflicts, information about what happened is often confusing, messy, and distributed across many sources. We would like to be able to automatically identify relevant information and assemble it into coherent narratives of what happened. To make this task accessible to neural models, we introduce Story Salads, mixtures of multiple documents that can be generated at scale. By exploiting the Wikipedia hierarchy, we can generate salads that exhibit challenging inference problems. Story salads give rise to a novel, challenging clustering task, where the objective is to group sentences from the same narratives. We demonstrate that simple bag-of-words similarity clustering falls short on this task and that it is necessary to take into account global context and coherence. (A) Some of the prisoners were survivors of the Battle of Qala-i-Jangi in Mazar-i-Sharif. (A) Chechnya came under the influence of warlords. (B) The U.S. invaded Afghanistan the same year when several Taliban prisoners were shot. (A) Russian federal troops entered Chechnya and ended its independence. (A) The Russian casualties included at least two commandos killed and 11 wounded. (B) The dead were buried in the same grave under the authority of Commander Kamal.2 In particular, while we do not focus on creating mixtures with conflicting information, it can often be found in mixtures created based on Wikipedia categories, as shown in Figure 2.

Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations

“…Several researchers have defined tasks related to discourse structure, including sentence ordering (Chen et al, 2016;Logeswaran et al, 2016;Cui et al, 2018), sentence clustering (Wang et al, 2018b), and disentangling textual threads (Elsner andCharniak, 2008, 2010;Lowe et al, 2015;Mehri and Carenini, 2017;Jiang et al, 2018;Kummerfeld et al, 2019).…”

“…Inspired by prior work on chat disentanglement (Elsner andCharniak, 2008, 2010) and sentence clustering (Wang et al, 2018b), we propose a sentence disentanglement task. The task is to determine whether a sequence of six sentences forms a coherent paragraph.…”

“…Our proposed dataset differs from the sentence clustering task of Wang et al (2018b) in that it preserves sentence order and does not anonymize or lemmatize words, because they play an important role in conveying information about discourse coherence.…”

Conditional Generation of Temporally-ordered Event Sequences

“…Another line of work instead directly focuses on extracting coherent narratives from "story salads" (Wang et al, 2018) or more broadly generate narratives given predefined scenarios (Wang et al, 2019;Qin et al, 2020). However, without considering temporal ordering, these systems are prone to learn discourse ordering of events instead of a strong representation of temporal knowledge.…”

Narrative Paths and Negotiation of Power in Birth Stories

“…One approach is the Narrative Cloze or Story Cloze Task, which given all but one missing section of a story asks for the correct insertion (e.g., the ending of the story) [20,72]. Another approach is to use mixtures of multiple stories and ask to pick apart the sentences from the separate stories [96]. We suggest that the structure of the birth stories collection provides a new testing ground for narrative analysis techniques.…”