Enriching the WebNLG corpus

Ferreira, Thiago Castro; Moussallem, Diego; Krahmer, Emiel; Wubben, Sander

doi:10.18653/v1/w18-6521

Cited by 28 publications

(37 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, while relation extraction datasets link text to a knowledge graph, the text is made up of disjoint sentences that do not provide sufficient context to train a powerful language model. Our goals are much more aligned to the data-to-text task (Ahn et al, 2016;Lebret et al, 2016;Wiseman et al, 2017;Yang et al, 2017;Gardent et al, 2017;Ferreira et al, 2018), where a small table-sized KB is provided to generate a short piece of text; we are interested in language models that dynamically decide the facts to incorporate from the knowledge graph, guided by the discourse. For these reasons we introduce the Linked WikiText-2 dataset, consisting of (approximately) the same articles appearing in the WikiText-2 language modeling corpus, but linked to the Wikidata (Vrandečić and Krötzsch, 2014) knowledge graph.…”

Section: Parameterizing the Distributionsmentioning

confidence: 99%

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling

Logan¹,

Liu²,

Peters³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

134

View full text Add to dashboard Cite

Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of remembering facts seen at training time, and often have difficulty recalling them. To address this, we introduce the knowledge graph language model (KGLM), a neural language model with mechanisms for selecting and copying facts from a knowledge graph that are relevant to the context. These mechanisms enable the model to render information it has never seen before, as well as generate out-of-vocabulary tokens. We also introduce the Linked WikiText-2 dataset, 1 a corpus of annotated text aligned to the Wikidata knowledge graph whose contents (roughly) match the popular WikiText-2 benchmark (Merity et al., 2017). In experiments, we demonstrate that the KGLM achieves significantly better performance than a strong baseline language model. We additionally compare different language models' ability to complete sentences requiring factual knowledge, and show that the KGLM outperforms even very large language models in generating facts.

show abstract

Section: Parameterizing the Distributionsmentioning

confidence: 99%

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling

Logan¹,

Liu²,

Peters³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

134

View full text Add to dashboard Cite

show abstract

“…We used version 1.5 of the augmented WebNLG corpus (Castro Ferreira et al, 2018b) 4 to evaluate the steps of our pipeline approach. Based on its intermediate representations, we extracted goldstandards to train and evaluate the different steps.…”

Section: Datamentioning

confidence: 99%

Neural data-to-text generation: A comparison between pipeline and end-to-end architectures

Ferreira

Lee

Miltenburg

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

Self Cite

View full text Add to dashboard Cite

Traditionally, most data-to-text applications have been designed using a modular pipeline architecture, in which non-linguistic input data is converted into natural language through several intermediate transformations. By contrast, recent neural models for data-to-text generation have been proposed as end-to-end approaches, where the non-linguistic input is rendered in natural language with much less explicit intermediate representations in between. This study introduces a systematic comparison between neural pipeline and endto-end data-to-text approaches for the generation of text from RDF triples. Both architectures were implemented making use of the encoder-decoder Gated-Recurrent Units (GRU) and Transformer, two state-of-the art deep learning methods. Automatic and human evaluations together with a qualitative analysis suggest that having explicit intermediate steps in the generation process results in better texts than the ones generated by end-to-end approaches. Moreover, the pipeline models generalize better to unseen inputs. Data and code are publicly available. 1

show abstract

“…We evaluated our proposal based on an enriched version (Castro Ferreira et al, 2018b) of the WebNLG corpus (Gardent et al, 2017a). The original resource is a parallel corpus with sets of RDF (Resource Description Framework) triples and their corresponding verbalizations.…”

Section: Datamentioning

confidence: 99%

“…We used an enriched version of the WebNLG corpus obtained by a delexicalization process (i.e., mapping each entity to a generic tag and later replacing their corresponding referring expressions in discourse with these tags) which was created by Castro Ferreira et al (2018b). Table 1 shows an example of a set of 4 triples and corresponding text, together with the intermediate representations obtained in the delexicalization process, such as general tags, Wikipedia IDs (entity/constant), referring expressions and the delexicalized template.…”

Section: Datamentioning

confidence: 99%

See 1 more Smart Citation

Referring to what you know and do not know: Making Referring Expression Generation Models Generalize To Unseen Entities

Cunha¹,

Ferreira

Pagano

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Data-to-text Natural Language Generation (NLG) is the computational process of generating natural language in the form of text or voice from non-linguistic data. A core micro-planning task within NLG is referring expression generation (REG), which aims to automatically generate noun phrases to refer to entities mentioned as discourse unfolds. A limitation of novel REG models is not being able to generate referring expressions to entities not encountered during the training process. To solve this problem, we propose two extensions to NeuralREG, a state-ofthe-art encoder-decoder REG model. The first is a copy mechanism, whereas the second consists of representing the gender and type of the referent as inputs to the model. Drawing on the results of automatic and human evaluation as well as an ablation study using the WebNLG corpus, we contend that our proposal contributes to the generation of more meaningful referring expressions to unseen entities than the original system and related work. Code and all produced data are publicly available 12 .

show abstract

Enriching the WebNLG corpus

Cited by 28 publications

References 19 publications

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling

Neural data-to-text generation: A comparison between pipeline and end-to-end architectures

Referring to what you know and do not know: Making Referring Expression Generation Models Generalize To Unseen Entities

Contact Info

Product

Resources

About