Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation

Abstract

Data-to-text generation can be conceptually divided into two parts: ordering and structuring the information (planning), and generating fluent language describing the information (realization). Modern neural generation systems conflate these two steps into a single end-to-end differentiable system. We propose to split the generation process into a symbolic text-planning stage that is faithful to the input, followed by a neural generation stage that focuses only on realization. For training a plan-to-text generator, we present a method for matching reference texts to their corresponding text plans. For inference time, we describe a method for selecting high-quality text plans for new inputs. We implement and evaluate our approach on the WebNLG benchmark. Our results demonstrate that decoupling text planning from neural realization indeed improves the system's reliability and adequacy while maintaining fluent output. We observe improvements both in BLEU scores and in manual evaluations. Another benefit of our approach is the ability to output diverse realizations of the same input, paving the way to explicit control over the generated text structure.

Survey of Hallucination in Natural Language Generation

“…Modeling and Inference Method. Planning or skeleton is a common method in data-to-text tasks to improve the faithfulness to the input [124]. Liu et al [107] propose a two-step generator with a separate text planner, which is augmented by auxiliary entity information.…”

RDF-to-Text Generation with Graph-augmented Structural Neural Encoders

“…To improve the probability of producing high-quality texts, Zhu et al [70] proposed a model minimizing the Kullback-Leibler (KL) divergence between the distributions of the real text and generated text. Moryossef et al [40] proposed a model combining the pipeline system and neural networks to match a reference text with its corresponding text plan to train a plan-to-text generator. In addition, some work has focused solely on improving the performance of the seen split part of dataset, where the entity types have appeared in the training set.…”

Improving Quality and Efficiency in Plan-based Neural Data-to-text Generation

“…1 In neural D2T, the common approaches train a neural end-to-end encoder-decoder system that encodes the input data and decodes an output text. In recent work (Moryossef et al, 2019) we proposed to adopt ideas from "traditional" language generation approaches (i.e. Reiter and Dale (2000); Walker et al (2007); Gatt and Krahmer (2017)) that separate the generation into a planning stage that determines the order and structure of the expressed facts, and a realization stage that maps the plan to natural language text.…”

“…In this work we adopt the step-by-step framework of Moryossef et al (2019) and propose four independent extensions that improve aspects of our original system: we suggest a new plan generation mechanism, based on a trainable-yetverifiable neural decoder, that is orders of magnitude faster than the original one ( §3); we use knowledge of the plan structure to add typing information to plan elements. This improves the system's performance on unseen relations and entities ( §4); the separation of planning from realizations allows the incorporation of a simple output verification heuristic that drastically improves the correctness of the output ( §5); and finally we incorporate a post-processing referring expression generation (REG) component, as proposed but not implemented in our previous work, to improve the naturalness of the resulting output ( §6).…”

“…

We follow the step-by-step approach to neural data-to-text generation we proposed in Moryossef et al (2019), in which the generation process is divided into a text-planning stage followed by a plan-realization stage. We suggest four extensions to that framework: (1) we introduce a trainable neural planning component that can generate effective plans several orders of magnitude faster than the original planner; (2) we incorporate typing hints that improve the model's ability to deal with unseen relations and entities; (3) we introduce a verification-by-reranking stage that substantially improves the faithfulness of the resulting texts; (4) we incorporate a simple but effective referring expression generation module.

…”