Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

Guo, Zhijiang; Zhang, Yan; Teng, Zhiyang; Lu, Wei

doi:10.1162/tacl_a_00269

Cited by 121 publications

(129 citation statements)

References 29 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…As a baseline (S2S), we train an attention-based encoderdecoder model with copy and coverage mechanisms, and use a linearized version of the graph generated by depth-first traversal order as input. We compare our models against several state-ofthe-art results reported on the two datasets (Konstas et al, 2017;Song et al, 2018;Beck et al, 2018;Damonte and Cohen, 2019;Cao and Clark, 2019;Guo et al, 2019).…”

Section: Resultsmentioning

confidence: 99%

“…Damonte and Cohen (2019) show that off-theshelf GCNs cannot achieve good performance for AMR-to-text generation. To tackle this issue, Guo et al (2019) introduce dense connectivity to GNNs in order to integrate both local and global features, achieving good results on the task. Our work is related to Damonte and Cohen (2019), that use stacking of GCN and LSTM layers to improve the model capacity and employ anonymization.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing AMR-to-Text Generation with Dual Graph Representations

Ribeiro¹,

Gardent

Gurevych³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Generating text from graph-based data, such as Abstract Meaning Representation (AMR), is a challenging task due to the inherent difficulty in how to properly encode the structure of a graph with labeled edges. To address this difficulty, we propose a novel graph-to-sequence model that encodes different but complementary perspectives of the structural information contained in the AMR graph. The model learns parallel top-down and bottom-up representations of nodes capturing contrasting views of the graph. We also investigate the use of different node message passing strategies, employing different state-of-the-art graph encoders to compute node representations based on incoming and outgoing perspectives. In our experiments, we demonstrate that the dual graph representation leads to improvements in AMR-to-text generation, achieving state-ofthe-art results on two AMR datasets 1 .

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Enhancing AMR-to-Text Generation with Dual Graph Representations

Ribeiro¹,

Gardent

Gurevych³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…LSTM 22.00 -GRN (Song et al, 2018) 23.28 -DCGCN (Guo et al, 2019) 25.70 -RA-Trans-SA (Zhu et al, 2019) Table 1: Main test results on LDC2015E86. Numbers such as "2M" means the number of extra silver data being used, and "ensemble" indicates model ensemble.…”

Section: Model Bleu Timementioning

confidence: 99%

Structural Information Preserving for Graph-to-Text Generation

Song¹,

Wang²,

Su³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

The task of graph-to-text generation aims at producing sentences that preserve the meaning of input graphs. As a crucial defect, the current state-of-the-art models may mess up or even drop the core structural information of input graphs when generating outputs. We propose to tackle this problem by leveraging richer training signals that can guide our model for preserving input information. In particular, we introduce two types of autoencoding losses, each individually focusing on different aspects (a.k.a. views) of input graphs. The losses are then back-propagated to better calibrate our model via multi-task training. Experiments on two benchmarks for graph-to-text generation show the effectiveness of our approach over a state-of-the-art baseline. Our code is available at http://github.com/ Soistesimmer/AMR-multiview.

show abstract

“…Data-to-text is the task of expressing the components (attributes and values) of meaning representation (MR) as human-readable natural sentences. Previous work in this area include templates (Reiter, 1995), rules (Reiter et al, 2005), pipelines (Reiter, 2007;Reiter and Dale, 1997), probabilistic models (Liang et al, 2009) and more recently end-to-end as well as neural-based methods (Wen et al, 2015;Mei et al, 2016;Dušek and Jurcicek, 2016;Lampouras and Vlachos, 2016;Dušek et al, 2020;Wiseman et al, 2017;Gong, 2018;Chen and Mooney, 2008;Reiter, 2017;Lebret et al, 2016;Distiawan et al, 2018;Gehrmann et al, 2018;Marcheggiani and Perez-Beltrachini, 2018;Guo et al, 2019b;Zhao et al, 2020). In our work, we use the state-of-the-art model from Zhao et al (2020) as our baseline.…”

Section: Data-to-text Generationmentioning

confidence: 99%

DORB: Dynamically Optimizing Multiple Rewards with Bandits

Pasunuru

Guo

Bansal

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Policy gradients-based reinforcement learning has proven to be a promising approach for directly optimizing non-differentiable evaluation metrics for language generation tasks. However, optimizing for a specific metric reward leads to improvements in mostly that metric only, suggesting that the model is gaming the formulation of that metric in a particular way without often achieving real qualitative improvements. Hence, it is more beneficial to make the model optimize multiple diverse metric rewards jointly. While appealing, this is challenging because one needs to manually decide the importance and scaling weights of these metric rewards. Further, it is important to consider using a dynamic combination and curriculum of metric rewards that flexibly changes over time. Considering the above aspects, in our work, we automate the optimization of multiple metric rewards simultaneously via a multi-armed bandit approach (DORB), where at each round, the bandit chooses which metric reward to optimize next, based on expected arm gains. We use the Exp3 algorithm for bandits and formulate two approaches for bandit rewards: (1) Single Multi-reward Bandit (SM-Bandit); (2) Hierarchical Multi-reward Bandit (HM-Bandit). We empirically show the effectiveness of our approaches via various automatic metrics and human evaluation on two important NLG tasks: question generation and data-to-text generation, including on an unseen-test transfer setup. Finally, we present interpretable analyses of the learned bandit curriculum over the optimized rewards.

show abstract

Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

Cited by 121 publications

References 29 publications

Enhancing AMR-to-Text Generation with Dual Graph Representations

Enhancing AMR-to-Text Generation with Dual Graph Representations

Structural Information Preserving for Graph-to-Text Generation

DORB: Dynamically Optimizing Multiple Rewards with Bandits

Contact Info

Product

Resources

About