Generating Diverse Story Continuations with Controllable Semantics

Abstract

We propose a simple and effective modeling framework for controlled generation of multiple, diverse outputs. We focus on the setting of generating the next sentence of a story given its context. As controllable dimensions, we consider several sentence attributes, including sentiment, length, predicates, frames, and automatically-induced clusters. Our empirical results demonstrate: (1) our framework is accurate in terms of generating outputs that match the target control values;(2) our model yields increased maximum metric scores compared to standard n-best list generation via beam search; (3) controlling generation with semantic frames leads to a stronger combination of diversity and quality than other control variables as measured by automatic metrics. We also conduct a human evaluation to assess the utility of providing multiple suggestions for creative writing, demonstrating promising results for the potential of controllable, diverse generation in a collaborative writing system.

Controlling Dialogue Generation with Semantic Exemplars

“…Semantics-Based Generation has reemerged for use in various tasks such as paraphrasing , machine translation (Marcheggiani et al, 2018) and story generation (Tu et al, 2019;Fan et al, 2019). Semantic representations such as semantic frames and semantic role labels provide abstractions that capture the underlying meanings of different surface realizations (e.g., paraphrases, other languages).…”

Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech

“…Despite the good performance of these models, one of their widely acknowledged intrinsic drawbacks is the generation of safe and commonplace responses (Sordoni et al, 2015) due to improper objective function (Li et al, 2016), lack of model variability (Serban et al, 2017;Zhao et al, 2017), weak conditional signal (Tao et al, 2018), and model over-confidence (Jiang and de Rijke, 2018). Such tendency has prompted the study of methods that improve diversity and has resulted in a wide variety of solutions, such as optimizing a different loss function (Li et al, 2016;, varying the latent space (Shao et al, 2019;, utilizing adversarial learning (Xu et al, 2018;Shetty et al, 2017;Shi et al, 2018), and leveraging non-conversational information (Wu et al, 2020;Su et al, 2020;Tu et al, 2019). Our work is different from all above in that we adopt a pipeline model which promotes diversity by generating a variety of candidates.…”

Changing the Mind of Transformers for Topically-Controllable Language Generation

“…The options could also be a single query word at the beginning (Austin, 2019), the article title (Yan, 2016), politeness (Niu and Bansal, 2018) or specificity (See et al, 2019b) of the text, or the length of the generated sentence (Tu et al, 2019). However, the options cannot provide fine-grained control on topical directions of the generated contents.…”