Neural data-to-text generation: A comparison between pipeline and end-to-end architectures

Abstract

Traditionally, most data-to-text applications have been designed using a modular pipeline architecture, in which non-linguistic input data is converted into natural language through several intermediate transformations. By contrast, recent neural models for data-to-text generation have been proposed as end-to-end approaches, where the non-linguistic input is rendered in natural language with much less explicit intermediate representations in between. This study introduces a systematic comparison between neural pipeline and endto-end data-to-text approaches for the generation of text from RDF triples. Both architectures were implemented making use of the encoder-decoder Gated-Recurrent Units (GRU) and Transformer, two state-of-the art deep learning methods. Automatic and human evaluations together with a qualitative analysis suggest that having explicit intermediate steps in the generation process results in better texts than the ones generated by end-to-end approaches. Moreover, the pipeline models generalize better to unseen inputs. Data and code are publicly available. 1

Best practices for the human evaluation of automatically generated text

“…text planning, surface realization, referring expression generation), with a concomitant focus on text output quality, for which an intrinsic evaluation may be sufficient. However, we are starting to see a swing back towards a full pipeline approach with separate neural modules handling sub-tasks (Castro Ferreira et al, 2019), which may also cause a resurgence of extrinsic evaluation.…”

Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019)

“…Specifically, for Chinese and Japanese, we require a proper method to tokenize/detokenize the results produced by our approach. Moreover, we aim to design the task based on novel pipeline architectures for Natural Language Generation (Ferreira et al, 2019…”

Controllable Meaning Representation to Text Generation: Linearization and Data Augmentation Strategies

“…Works like Moryossef et al (2019a,b) and Castro Ferreira et al (2019) show that treating various planning tasks as separate components in a pipeline, where the components themselves are implemented with neural models, improves the overall quality and semantic correctness of generated utterances relative to a completely end-to-end neural NLG model. However, they do not test the systematicty of the neural generation components, i.e.…”