Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning

Peng, Xiangyu; Li, Siyan; Wiegreffe, Sarah; Riedl, Mark O.

doi:10.48550/arxiv.2105.01311

Cited by 4 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Commonsense reasoning can also help story generation with issues in plot coherence. The Commonsense inference Augmented neural Sto-ryTelling (CAST) framework (Peng et al, 2021) introduces commonsense into the generation process, explicitly modeling interactions between multiple characters. The stricter, more explicit generation constraints of CAST produce more coherent and on-topic two-character stories that generating via sampling from a distribution alone.…”

Section: Commonsense Inference For Storytellingmentioning

confidence: 99%

$\rm{C {\small IS}}^2$: A Simplified Commonsense Inference Evaluation for Story Prose

Li¹,

Martin²,

Callison-Burch³

2022

Preprint

View full text Add to dashboard Cite

Transformers have been showing near-human performance on a variety of tasks, but they are not without their limitations. We discuss the issue of conflating results of transformers that are instructed to do multiple tasks simultaneously. In particular, we focus on the domain of commonsense reasoning within story prose, which we call contextual commonsense inference (CCI). We look at the GLU-COSE dataset and task for predicting implicit commonsense inferences between story sentences. Since the GLUCOSE task simultaneously generates sentences and predicts the CCI relation, there is a conflation in the results. Is the model really measuring CCI or is its ability to generate grammatical text carrying the results? In this paper, we introduce the task contextual commonsense inference in sentence selection (CIS 2 ), a simplified task that avoids conflation by eliminating language generation altogether. Our findings emphasize the necessity of future work to disentangle language generation from the desired NLP tasks at hand.

show abstract

Section: Commonsense Inference For Storytellingmentioning

confidence: 99%

$\rm{C {\small IS}}^2$: A Simplified Commonsense Inference Evaluation for Story Prose

Li¹,

Martin²,

Callison-Burch³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While such representations can be very useful, they have limited expressivity, are difficult to make for book-length stories, and do not capture much nuance. Some modern models, like CAST (Peng et al, 2021), which aim to emulate a reader by creating a graphical representation of the story, suffer from similar issues.…”

Section: Representations Of Knowledge In Computational Narratologymentioning

confidence: 99%

Cut the CARP: Fishing for zero-shot story evaluation

Matiana¹,

Smith²,

Teehan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in large-scale language models (Raffel et al., 2019;Brown et al., 2020) have brought significant qualitative and quantitative improvements in machine-driven text generation. Despite this, generation and evaluation of machine-generated narrative text remains a challenging problem. Objective evaluation of computationally-generated stories may be prohibitively expensive, require meticulously annotated datasets, or may not adequately measure the logical coherence of a generated story's narratological structure. Informed by recent advances in contrastive learning (Radford et al., 2021), we present Contrastive Authoring and Reviewing Pairing (carp): a scalable, efficient method for performing qualitatively superior, zero-shot evaluation of stories. We show a strong correlation between human evaluation of stories and those of carp. Model outputs more significantly correlate with corresponding human input than those language-model based methods which utilize finetuning or prompt engineering approaches. We also present and analyze the Story-Critique Dataset, a new corpora composed of 1.3 million aligned story-critique pairs derived from over 80,000 stories. We expect this corpus to be of interest to NLP researchers.

show abstract

“…Thus the resulting language model, named COMET (Bosselut et al, 2019;Hwang et al, 2021), becomes capable of inferring new commonsense knowledge on novel phrases. Ammanabrolu et al (2021b) proposes Causal, Commonsense Plot Ordering (C2PO) framework which takes advantage of COMET to infer predecessor and successor events and then bi-directionally search from prespecified start event to end event, however, C2PO generates plots made up of highly constrained, templated text; Peng et al (2021) leverages COMET to infer the character intentions and effects of actions so as to guide the generation process, but they did not consider controllability. There are also other approaches that directly incorporate commonsense knowledge graphs into the encoding process (Mihaylov and Frank, 2018;Guan et al, 2019).…”

Section: Related Work and Backgroundmentioning

confidence: 99%

“…In this paper, we evaluate coherence using human participant evaluation, asking a set of questions that includes dimensions such a logical coherence, loyalty to plot, and enjoyability. Variations of these questions have been used to evaluate other story generation systems (Purdy et al, 2018;Tambwekar et al, 2019;Ammanabrolu et al, 2020Ammanabrolu et al, , 2021aCastricato et al, 2021;Peng et al, 2021). We focus on dimensions involving overall perceptions of narrative coherence:…”

Section: Story Coherence Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Guiding Neural Story Generation with Reader Models

Peng¹,

Xie²,

Alabdulkarim³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Automated storytelling has long captured the attention of researchers for the ubiquity of narratives in everyday life. However, it is challenging to maintain coherence and stay ontopic toward a specific ending when generating narratives with neural language models. In this paper, we introduce Story generation with Reader Models (StoRM), a framework in which a reader model is used to reason about the story should progress. A reader model infers what a human reader believes about the concepts, entities, and relations about the fictional story world. We show how an explicit reader model represented as a knowledge graph affords story coherence and provides controllability in the form of achieving a given story world state goal. Experiments show that our model produces significantly more coherent and on-topic stories, outperforming baselines in dimensions including plot plausibility and staying on topic. Our system also outperforms outline-guided story generation baselines in composing given concepts without ordering.

show abstract

Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning

Cited by 4 publications

References 19 publications

$\rm{C {\small IS}}^2$: A Simplified Commonsense Inference Evaluation for Story Prose

$\rm{C {\small IS}}^2$: A Simplified Commonsense Inference Evaluation for Story Prose

Cut the CARP: Fishing for zero-shot story evaluation

Guiding Neural Story Generation with Reader Models

Contact Info

Product

Resources

About