Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation

Abstract

AMR-to-text generation is used to transduce Abstract Meaning Representation structures (AMR) into text. A key challenge in this task is to efficiently learn effective graph representations. Previously, Graph Convolution Networks (GCNs) were used to encode input AMRs, however, vanilla GCNs are not able to capture non-local information and additionally, they follow a local (first-order) information aggregation scheme. To account for these issues, larger and deeper GCN models are required to capture more complex interactions. In this paper, we introduce a dynamic fusion mechanism, proposing Lightweight Dynamic Graph Convolutional Networks (LDGCNs) that capture richer non-local interactions by synthesizing higher order information from the input graphs. We further develop two novel parameter saving strategies based on the group graph convolutions and weight tied convolutions to reduce memory usage and model complexity. With the help of these strategies, we are able to train a model with fewer parameters while maintaining the model capacity. Experiments demonstrate that LDGCNs outperform stateof-the-art models on two benchmark datasets for AMR-to-text generation with significantly fewer parameters. * * Equally Contributed. Work done while Yan Zhang was an intern at DAMO Academy, Alibaba Group and Zhijiang Guo was at the University of Edinburgh.

Structural Adapters in Pretrained Language Models for AMR-to-text Generation

“…In LDC2017T10, STRUCTADAPT-RGCN achieves a BLEU score of 44.03 training only 6.3% of the T5 base parameters, outperforming Ribeiro et al (2020b) which fine-tunes T5 updating significantly more parameters. STRUCTADAPT also achieves stateof-the-art performance on LDC2020T02, considerably improving over Zhang et al (2020b). Overall, the results indicate that explicitly considering the graph structure using an adapter mechanism is effective for AMR-to-text generation, significantly reducing the number of trained parameters while improving generation quality.…”

“…3 Dev BLEU is used for model selection. Following recent work in AMR-to-text generation (Mager et al, 2020;Zhang et al, 2020b), we use LDC2017T10 and LDC2020T02 corpora for evaluation of the proposed model. An instance in both datasets consists of a sentence annotated with its corresponding AMR graph.…”

“…Fine-tuning for Graph-to-text generation While previous approaches (Song et al, 2018;Ribeiro et al, 2019;Cai and Lam, 2020;Schmitt et al, 2020;Zhang et al, 2020b) have shown that explicitly encoding the graph structure is beneficial, fine-tuning PLMs on linearized structured data has established a new level of performance in data-to-text generation (Radev et al, 2020;Kale, 2020). Our work can be seen as integrating the advantage of both graph structure encoding and PLMs, using a novel adapter module.…”

Structural Adapters in Pretrained Language Models for AMR-to-Text Generation

“…BLEU is used for the stopping criterion. Following recent work (Mager et al, 2020;Zhang et al, 2020b), we evaluate our proposed models on LDC2017T10 and LDC2020T02 corpora.…”

“…Fine-tuning for Graph-to-text Generation. While previous approaches (Song et al, 2018;Ribeiro et al, 2019;Cai and Lam, 2020;Schmitt et al, 2021;Zhang et al, 2020b) have shown that explicitly encoding the graph structure is beneficial, fine-tuning PLMs on linearized structured data has established a new level of performance in data-to-text generation (Nan et al, 2021;Kale, 2020;. Our work can be seen as integrating the advantage of both graph structure encoding and PLMs, using a novel adapter module.…”