Controllable Neural Story Plot Generation via Reward Shaping

Abstract

Language-modeling-based approaches to story plot generation attempt to construct a plot by sampling from a language model (LM) to predict the next character, word, or sentence to add to the story. LM techniques lack the ability to receive guidance from the user to achieve a specific goal, resulting in stories that don't have a clear sense of progression and lack coherence. We present a reward-shaping technique that analyzes a story corpus and produces intermediate rewards that are backpropagated into a pre-trained LM in order to guide the model towards a given goal. Automated evaluations show our technique can create a model that generates story plots which consistently achieve a specified goal. Human-subject studies show that the generated stories have more plausible event ordering than baseline plot generation techniques.

Proceedings of the Second Workshop on Storytelling

“…It is, therefore, important, that this observation finds its place in automatic storytelling systems. Some attempts have been done in natural language generation towards controllable story generation Tambwekar et al, 2018). We propose that emotion expression should be one of the controllable parameters in automatic storytellers.…”

“…Our event-to-event system is the policy gradient deep reinforcement learner from Tambwekar et al (2019). Briefly, the technique starts with a sequence-to-sequence LSTM model trained to perform the event-to-event task.…”

“…The reward is the product of the distance of each verb from the goal verb by the normalized frequency of how often the verb occurs before the goal verb in stories. Details of how the reward is calculated are given in Tambwekar et al (2019).…”

“…We scraped long-running science fiction TV show plot summaries from the fandom wiki service wikia.com. This fandom wiki service contains longer and more detailed plot summaries than the dataset used in Tambwekar et al (2019), both of which are qualities that we believe to be important for the overall story generation process. The corpus contains 2,276 stories in total, each an episode of a TV show.…”

ExpReal: a Writing Language and System for Authoring Texts in Interactive Narrative

“…These structures then need to be further processed to produce a text in natural language, serving either as textual description of the action or as characters' dialogue lines. This two-step generation process is common to very different approaches such as character-based planning [2], rule-based forward simulation of narrative acts [15], broad autonomous agents [6], and even machine learning based approaches [16]. Because the first step is the biggest challenge in the field of Interactive Narrative, the second step tends to be neglected.…”