Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence

Abstract

The neural attention model has achieved great success in data-to-text generation tasks.Though usually excelling at producing fluent text, it suffers from the problem of information missing, repetition and "hallucination". Due to the black-box nature of the neural attention architecture, avoiding these problems in a systematic way is non-trivial. To address this concern, we propose to explicitly segment target text into fragment units and align them with their data correspondences. The segmentation and correspondence are jointly learned as latent variables without any human annotations. We further impose a soft statistical constraint to regularize the segmental granularity. The resulting architecture maintains the same expressive power as neural attention models, while being able to generate fully interpretable outputs with several times less computational cost. On both E2E and WebNLG benchmarks, we show the proposed model consistently outperforms its neural attention counterparts.

Stage-wise Fine-tuning for Graph-to-Text Generation

“…Previous methods usually treat the graphto-text generation as an end-to-end generation task. Those models (Trisedya et al, 2018;Gong et al, 2019;Shen et al, 2020) usually first lineralize the knowledge graph and then use attention mechanism to generate the description sentences. While the linearization of input graph may sacrifice the inter-dependency inside input graph, some papers (Ribeiro et al, 2019(Ribeiro et al, , 2020aZhao et al, 2020) Category Output use graph encoder such as GCN (Duvenaud et al, 2015) and graph transformer Koncel-Kedziorski et al, 2019) to encode the input graphs.…”

“…While the linearization of input graph may sacrifice the inter-dependency inside input graph, some papers (Ribeiro et al, 2019(Ribeiro et al, , 2020aZhao et al, 2020) Category Output use graph encoder such as GCN (Duvenaud et al, 2015) and graph transformer Koncel-Kedziorski et al, 2019) to encode the input graphs. Others (Shen et al, 2020; try to carefully design loss functions to control the generation quality. With the development of computation resources, large scale PLMs such as GPT-2 (Radford et al, 2019), BART (Lewis et al, 2020) and T5 (Raffel et al, 2020) achieve state-ofthe-art results even with simple linearized graph input (Harkous et al, 2020;Chen et al, 2020a;Kale, 2020;Ribeiro et al, 2020b).…”

AggGen: Ordering and Aggregating while Generating

“…However it trades the controllability for fluency. Similarly, Shen et al (2020) explicitly segment target text into fragment units, while aligning them with their corresponding input. Shao et al (2019) use a Hierarchical Variational Model to aggregate input items into a sequence of local latent variables and realize sentences conditioned on the aggregations.…”

“…In contrast, the plan used in other neural plan-based approaches is usually limited in terms of its interpretability, control, and expressivity. For example, in (Moryossef et al, 2019b;Zhao et al, 2020) the sentence plan is created independently, incurring error propagation; Wiseman et al (2018) use latent segmentation that limits interpretability; Shao et al (2019) sample from a latent variable, not allowing for explicit control; and Shen et al (2020) aggregate multiple input representations which limits expressiveness. AGGGEN explicitly models the two planning processes (ordering and aggregation), but can directly influence the resulting plan and generated target text, using a separate inference algorithm based on dynamic programming.…”

Evaluating Factuality in Generation with Dependency-level Entailment

“…The rise of pre-trained language models (Devlin et al, 2019;Radford et al, 2019) has led to strong text generation models for applications including summarization (Dong et al, 2019;, paraphrasing (Goyal and Durrett, 2020;Shen et al, 2020), story generation (Mao et al, 2019), and data augmentation Zhang and Bansal, 2019). However, while these models generate fluent and grammatical text, they are prone to making factual errors that contradict…”