Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation

Zhao, Chao; Walker, Marilyn A.; Chaturvedi, Snigdha

doi:10.18653/v1/2020.acl-main.224

Cited by 68 publications

(70 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 presents our baselines on the WebNLG data-to-text task. Our cross-entropy model is comparable to the very recent state-of-theart model (Zhao et al, 2020). Further, we present single reward based RL models with ROUGE-L, BLEU, and Entailment score as rewards, which again perform better than our cross-entropy model.…”

Section: Results On Data-to-text Generationmentioning

confidence: 61%

See 1 more Smart Citation

DORB: Dynamically Optimizing Multiple Rewards with Bandits

Pasunuru

Guo

Bansal

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Policy gradients-based reinforcement learning has proven to be a promising approach for directly optimizing non-differentiable evaluation metrics for language generation tasks. However, optimizing for a specific metric reward leads to improvements in mostly that metric only, suggesting that the model is gaming the formulation of that metric in a particular way without often achieving real qualitative improvements. Hence, it is more beneficial to make the model optimize multiple diverse metric rewards jointly. While appealing, this is challenging because one needs to manually decide the importance and scaling weights of these metric rewards. Further, it is important to consider using a dynamic combination and curriculum of metric rewards that flexibly changes over time. Considering the above aspects, in our work, we automate the optimization of multiple metric rewards simultaneously via a multi-armed bandit approach (DORB), where at each round, the bandit chooses which metric reward to optimize next, based on expected arm gains. We use the Exp3 algorithm for bandits and formulate two approaches for bandit rewards: (1) Single Multi-reward Bandit (SM-Bandit); (2) Hierarchical Multi-reward Bandit (HM-Bandit). We empirically show the effectiveness of our approaches via various automatic metrics and human evaluation on two important NLG tasks: question generation and data-to-text generation, including on an unseen-test transfer setup. Finally, we present interpretable analyses of the learned bandit curriculum over the optimized rewards.

show abstract

Section: Results On Data-to-text Generationmentioning

confidence: 61%

“…Given a set of Resource Description Framework (RDF) triples, 4 the task is to generate a natural language text describing the facts in the RDF data. Following Zhao et al (2020), we serialize and reorder the RDF data as an intermediate planning setup, and feed the plan into a seq2seq model with attention and copy mechanism.…”

Section: Data-to-text Generationmentioning

confidence: 99%

DORB: Dynamically Optimizing Multiple Rewards with Bandits

Pasunuru

Guo

Bansal

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Those models (Trisedya et al, 2018;Gong et al, 2019;Shen et al, 2020) usually first lineralize the knowledge graph and then use attention mechanism to generate the description sentences. While the linearization of input graph may sacrifice the inter-dependency inside input graph, some papers (Ribeiro et al, 2019(Ribeiro et al, , 2020aZhao et al, 2020) Category Output use graph encoder such as GCN (Duvenaud et al, 2015) and graph transformer Koncel-Kedziorski et al, 2019) to encode the input graphs. Others (Shen et al, 2020; try to carefully design loss functions to control the generation quality.…”

Section: Related Workmentioning

confidence: 99%

Stage-wise Fine-tuning for Graph-to-Text Generation

Wang¹,

Yavuz²,

Lin³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Graph-to-text generation has benefited from pre-trained language models (PLMs) in achieving better performance than structured graph encoders. However, they fail to fully utilize the structure information of the input graph. In this paper, we aim to further improve the performance of the pre-trained language model by proposing a structured graph-to-text model with a two-step fine-tuning mechanism which first fine-tunes the model on Wikipedia before adapting to the graph-to-text generation. In addition to using the traditional token and position embeddings to encode the knowledge graph (KG), we propose a novel treelevel embedding method to capture the interdependency structures of the input graph. This new approach has significantly improved the performance of all text generation metrics for the English WebNLG 2017 dataset. 1 * This research was conducted during the author's internship at Salesforce Research.

show abstract

“…Translation models take natural language input and must faithfully decode into natural language output. However, as shown in Zhao et al (2020), bridging the gap between structured input and linear output is a difficult task. In addition, in structured input such as graphs, the input is usually semantically under-specified.…”

Section: Introductionmentioning

confidence: 99%

Multilingual AMR-to-Text Generation

Fan¹,

Gardent²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Generating text from structured data is challenging because it requires bridging the gap between (i) structure and natural language (NL) and (ii) semantically underspecified input and fully specified NL output. Multilingual generation brings in an additional challenge: that of generating into languages with varied word order and morphological properties. In this work, we focus on Abstract Meaning Representations (AMRs) as structured input, where previous research has overwhelmingly focused on generating only into English. We leverage advances in cross-lingual embeddings, pretraining, and multilingual models to create multilingual AMR-to-text models that generate in twenty one different languages. For eighteen languages, based on automatic metrics, our multilingual models surpass baselines that generate into a single language. We analyse the ability of our multilingual models to accurately capture morphology and word order using human evaluation, and find that native speakers judge our generations to be fluent.

show abstract

Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation

Cited by 68 publications

References 33 publications

DORB: Dynamically Optimizing Multiple Rewards with Bandits

DORB: Dynamically Optimizing Multiple Rewards with Bandits

Stage-wise Fine-tuning for Graph-to-Text Generation

Multilingual AMR-to-Text Generation

Contact Info

Product

Resources

About