Strategies for Structuring Story Generation

Fan, Angela; Lewis, Mike; Dauphin, Yann N.

doi:10.18653/v1/p19-1254

Cited by 169 publications

(171 citation statements)

References 31 publications

Supporting

Mentioning

153

Contrasting

Order By: Relevance

“…Recent studies shows that the modeling ability of multi-head attention has not been completely developed. Several specific guidance cues of different heads without breaking the vanilla multi-head attention mechanism can further boost the performance, e.g., disagreement regularization (Li et al, 2018;Tao et al, 2018), information aggregation (Li et al, 2019a), and functional specialization (Fan et al, 2019) on attention heads, the combination of multi-head attention with multi-task learning (Strubell et al, 2018). Our work demonstrates that multi-head attention also benefits from the integration of the phrase information.…”

Section: Related Workmentioning

confidence: 72%

Multi-Granularity Self-Attention for Neural Machine Translation

Hao¹,

Wang²,

Shi³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Current state-of-the-art neural machine translation (NMT) uses a deep multi-head selfattention network with no explicit phrase information. However, prior work on statistical machine translation has shown that extending the basic translation unit from words to phrases has produced substantial improvements, suggesting the possibility of improving NMT performance from explicit modeling of phrases. In this work, we present multi-granularity self-attention (MG-SA): a neural network that combines multi-head selfattention and phrase modeling. Specifically, we train several attention heads to attend to phrases in either n-gram or syntactic formalism. Moreover, we exploit interactions among phrases to enhance the strength of structure modeling -a commonly-cited weakness of self-attention. Experimental results on WMT14 English-to-German and NIST Chinese-to-English translation tasks show the proposed approach consistently improves performance. Targeted linguistic analysis reveals that MG-SA indeed captures useful phrase information at various levels of granularities.

show abstract

Section: Related Workmentioning

confidence: 72%

Multi-Granularity Self-Attention for Neural Machine Translation

Hao¹,

Wang²,

Shi³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…Other work has looked for innovative ways to separate planning and surface realization from the end-to-end neural systems, most notably Wiseman et al (2018) which learns template generation also on the E2E task, but does not yet match baseline performance, and He et al (2018) which has a dialogue manager control decision making and passes this information onto a secondary language generator. Other work has attempted either multi-stage semi-unconstrained language generation, such as in the domain of story telling (Fan et al, 2019), or filling-in-the-blanks style sentence reconstruction (Fedus et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Designing a Symbolic Intermediate Representation for Neural Surface Realization

Elder¹,

Foster²,

Barry³

et al. 2019

Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation

View full text Add to dashboard Cite

Generated output from neural NLG systems often contain errors such as hallucination, repetition or contradiction. This work focuses on designing a symbolic intermediate representation to be used in multi-stage neural generation with the intention of reducing the frequency of failed outputs. We show that surface realization from this intermediate representation is of high quality and when the full system is applied to the E2E dataset it outperforms the winner of the E2E challenge. Furthermore, by breaking out the surface realization step from typically end-to-end neural systems, we also provide a framework for non-neural content selection and planning systems to potentially take advantage of semi-supervised pretraining of neural surface realization models.

show abstract

“…For instance, [9] generate a coherent story from independent descriptions, describing a scene or an event. [5] explore a strategy for story generation. Their frameworks both use sequence-to-sequence neural networks.…”

Section: Related Work 21 Story Generationmentioning

confidence: 99%

Controllable Multi-Character Psychology-Oriented Story Generation

Wang

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Story generation, which aims to generate a long and coherent story automatically based on the title or an input sentence, is an important research area in the field of natural language generation. There is relatively little work on story generation with appointed emotions. Most existing works focus on using only one specific emotion to control the generation of a whole story and ignore the emotional changes in the characters in the course of the story. In our work, we aim to design an emotional line for each character that considers multiple emotions common in psychological theories, with the goal of generating stories with richer emotional changes in the characters. To the best of our knowledge, this work is first to focuses on characters' emotional lines in story generation. We present a novel model-based attention mechanism that we call SoCP (Storytelling of multi-Character Psychology). We show that the proposed model can generate stories considering the changes in the psychological state of different characters. To take into account the particularity of the model, in addition to commonly used evaluation indicators(BLEU, ROUGE, etc.), we introduce the accuracy rate of psychological state control as a novel evaluation metric. The new indicator reflects the effect of the model on the psychological state control of story characters. Experiments show that with SoCP, the generated stories follow the psychological state for each character according to both automatic and human evaluations. CCS CONCEPTS • Computing methodologies → Natural language generation.

show abstract

Strategies for Structuring Story Generation

Cited by 169 publications

References 31 publications

Multi-Granularity Self-Attention for Neural Machine Translation

Multi-Granularity Self-Attention for Neural Machine Translation

Designing a Symbolic Intermediate Representation for Neural Surface Realization

Controllable Multi-Character Psychology-Oriented Story Generation

Contact Info

Product

Resources

About