Findings of the E2E NLG Challenge

Dušek, Ondřej; Novikova, Jekaterina; Rieser, Verena

doi:10.18653/v1/w18-6539

Cited by 88 publications

(88 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We apply pragmatics to encourage output strings from which the input MR can be identified. For our S 0 model, we use a publicly-released neural generation system (Puzikov and Gurevych, 2018) that achieves comparable performance to the best published results in Dušek et al (2018). Abstractive Summarization Our second task is multi-sentence document summarization.…”

Section: Meaning Representationsmentioning

confidence: 99%

“…We report the task's five automatic metrics: BLEU (Papineni et al, 2002), NIST (Doddington, 2002), METEOR (Lavie and Agarwal, 2007), ROUGE-L (Lin, 2004) and CIDEr (Vedantam et al, 2015). Table 1 compares the performance of our base S 0 and pragmatic models to the baseline T-Gen system (Dušek and Jurčíček, 2016) and the best previous result from the 20 primary systems evaluated in the E2E challenge (Dušek et al, 2018). The systems obtaining these results encompass a range of approaches: a template system (Puzikov and Gurevych, 2018), a neural model (Zhang et al, 2018), models trained with reinforcement learning (Gong, 2018), and systems using ensembling and reranking (Juraska et al, 2018).…”

Section: Meaning Representationsmentioning

confidence: 99%

See 1 more Smart Citation

Pragmatically Informative Text Generation

Shen¹,

Fried²,

Andreas³

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

We improve the informativeness of models for conditional text generation using techniques from computational pragmatics. These techniques formulate language production as a game between speakers and listeners, in which a speaker should generate output text that a listener can use to correctly identify the original input that the text describes. While such approaches are widely used in cognitive science and grounded language learning, they have received less attention for more standard language generation tasks. We consider two pragmatic modeling methods for text generation: one where pragmatics is imposed by information preservation, and another where pragmatics is imposed by explicit modeling of distractors. We find that these methods improve the performance of strong existing systems for abstractive summarization and generation from structured meaning representations.

show abstract

Section: Meaning Representationsmentioning

confidence: 99%

Section: Meaning Representationsmentioning

confidence: 99%

Pragmatically Informative Text Generation

Shen¹,

Fried²,

Andreas³

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

show abstract

“…1 Our model performs sentencelevel content planning for information selection and ordering, and style-controlled surface realization to produce the final generation. We focus on conditional text generation problems (Lebret et al, 2016;Colin et al, 2016;Dušek et al, 2018): As shown in Figure 2, the input to our model consists of a topic statement and a set of keyphrases. The output is a relevant and coherent paragraph to reflect the salient points from the input.…”

Section: Simple Wikipediamentioning

confidence: 99%

Sentence-Level Content Planning and Style Specification for Neural Text Generation

Hua¹,

Wang²

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Building effective text generation systems requires three critical components: content selection, text planning, and surface realization, and traditionally they are tackled as separate problems. Recent all-in-one style neural generation models have made impressive progress, yet they often produce outputs that are incoherent and unfaithful to the input. To address these issues, we present an end-toend trained two-step generation model, where a sentence-level content planner first decides on the keyphrases to cover as well as a desired language style, followed by a surface realization decoder that generates relevant and coherent text. For experiments, we consider three tasks from domains with diverse topics and varying language styles: persuasive argument construction from Reddit, paragraph generation for normal and simple versions of Wikipedia, and abstract generation for scientific articles. Automatic evaluation shows that our system can significantly outperform competitive comparisons. Human judges further rate our system generated text as more fluent and correct, compared to the generations by its variants that do not consider language style.

show abstract

“…Traditionally, these two subproblems have been tackled separately. In recent years, neural generation models, especially the encoder-decoder model, solve these two subproblems jointly and have achieved remarkable successes in several benchmarks (Mei et al, 2016;Lebret et al, 2016;Wiseman et al, 2017;Dušek et al, 2018;Nie et al, 2018). Such end-to-end data-to-text models rely on massive parallel pairs of data and text to learn the writing knowledge.…”

Section: Infoboxmentioning

confidence: 99%

Enhancing Neural Data-To-Text Generation Models with External Background Knowledge

Chen

Wang

Feng

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Recent neural models for data-to-text generation rely on massive parallel pairs of data and text to learn the writing knowledge. They often assume that writing knowledge can be acquired from the training data alone. However, when people are writing, they not only rely on the data but also consider related knowledge. In this paper, we enhance neural data-totext models with external knowledge in a simple but effective way to improve the fidelity of generated text. Besides relying on parallel data and text as in previous work, our model attends to relevant external knowledge, encoded as a temporary memory, and combines this knowledge with the context representation of data before generating words. This allows the model to infer relevant facts which are not explicitly stated in the data table from an external knowledge source. Experimental results on twenty-one Wikipedia infoboxto-text datasets show our model, KBAtt, consistently improves a state-of-the-art model on most of the datasets. In addition, to quantify when and why external knowledge is effective, we design a metric, KBGain, which shows a strong correlation with the observed performance boost. This result demonstrates the relevance of external knowledge and sparseness of original data are the main factors affecting system performance.

show abstract

Findings of the E2E NLG Challenge

Cited by 88 publications

References 29 publications

Pragmatically Informative Text Generation

Pragmatically Informative Text Generation

Sentence-Level Content Planning and Style Specification for Neural Text Generation

Enhancing Neural Data-To-Text Generation Models with External Background Knowledge

Contact Info

Product

Resources

About