Online Back-Parsing for AMR-to-Text Generation

Bai, Xuefeng; Song, Linfeng; Zhang, Yue

doi:10.18653/v1/2020.emnlp-main.92

Cited by 14 publications

(14 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior work, involving both GNNs and pretrained linearized models, has explored various ways of improving models' sensitivity to the structure of the input graph. To better maintain fidelity to the graph, previous graph-to-text methods incorporate additional loss terms, specialized architectures, or generation-time ranking to influence the semantic accuracy of generation: ranking outputs by the correctness of the AMR parse (Mager et al, 2020;Harkous et al, 2020), jointly "back-parsing" graphs when decoding (Bai et al, 2020), or using distinct components to model different graph traversals (Ribeiro et al, 2019).…”

Section: Rq2: Better Implicit Graph Encodings With Text-to-text Scaffoldingmentioning

confidence: 99%

Promoting Graph Awareness in Linearized Graph-to-Text Generation

Alexander¹,

Marasovi²,

Smith³

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Generating text from structured inputs, such as meaning representations or RDF triples, has often involved the use of specialized graphencoding neural networks. However, recent applications of pretrained transformers to linearizations of graph inputs have yielded stateof-the-art generation results on graph-to-text tasks. Here, we explore the ability of these linearized models to encode local graph structures, in particular their invariance to the graph linearization strategy and their ability to reconstruct corrupted inputs. Our findings motivate solutions to enrich the quality of models' implicit graph encodings via scaffolding. Namely, we use graph-denoising objectives implemented in a multi-task text-to-text framework. We find that these denoising scaffolds lead to substantial improvements in downstream generation in low-resource settings.

show abstract

Section: Rq2: Better Implicit Graph Encodings With Text-to-text Scaffoldingmentioning

confidence: 99%

Promoting Graph Awareness in Linearized Graph-to-Text Generation

Alexander¹,

Marasovi²,

Smith³

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

show abstract

“…However, seq2seq approaches tend to lose structural information in AMR graphs since they simply linearize AMR graphs into sequences before feeding them into the models. To prevent information loss caused by linearization, a variety of graph-tosequence approaches have been proposed to better model structural information (Song et al, 2018;Beck et al, 2018;Damonte and Cohen, 2019;Guo et al, 2019;Ribeiro et al, 2019;Cai and Lam, 2020b;Zhao et al, 2020;Yao et al, 2020;Bai et al, 2020). By taking advantages of strong pre-trained language models, recent studies achieve new state of the art (Mager et al, 2020;Harkous et al, 2020;Ribeiro et al, 2020;Bevilacqua et al, 2021) .…”

Section: Related Workmentioning

confidence: 99%

XLPT-AMR: Cross-Lingual Pre-Training via Multi-Task Learning for Zero-Shot AMR Parsing and Text Generation

Xu¹,

Zhu

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Due to the scarcity of annotated data, Abstract Meaning Representation (AMR) research is relatively limited and challenging for languages other than English. Upon the availability of English AMR dataset and English-to-X parallel datasets, in this paper we propose a novel cross-lingual pre-training approach via multi-task learning (MTL) for both zeroshot AMR parsing and AMR-to-text generation. Specifically, we consider three types of relevant tasks, including AMR parsing, AMR-to-text generation, and machine translation. We hope that knowledge gained while learning for English AMR parsing and text generation can be transferred to the counterparts of other languages. With properly pretrained models, we explore four different finetuning methods, i.e., vanilla fine-tuning with a single task, one-for-all MTL fine-tuning, targeted MTL fine-tuning, and teacher-studentbased MTL fine-tuning. Experimental results on AMR parsing and text generation of multiple non-English languages demonstrate that our approach significantly outperforms a strong baseline of pre-training approach, and greatly advances the state of the art. In detail, on LDC2020T07 we have achieved 70.45%, 71.76%, and 70.80% in Smatch F1 for AMR parsing of German, Spanish, and Italian, respectively, while for AMR-to-text generation of the languages, we have obtained 25.69, 31.36, and 28.42 in BLEU respectively. We make our code available on github https:// github.com/xdqkid/XLPT-AMR.

show abstract

“…machine translation (Song et al, 2019). Both statistical (Flanigan et al, 2016;Pourdamghani et al, 2016) and neural methods (Bai et al, 2020;Cai and Lam, 2020) have been investigated for AMRto-text generation, and recent methods make use of graph neural networks (GNNs) (Kipf and Welling, 2017) or Transformers (Vaswani et al, 2017) for representing the input graph.…”

Section: Amr Graphmentioning

confidence: 99%

Structural Adapters in Pretrained Language Models for AMR-to-text Generation

Ribeiro¹,

Zhang²,

Gurevych³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Previous work on text generation from graphstructured data relies on pretrained language models (PLMs) and utilizes graph linearization heuristics rather than explicitly considering the graph structure. Efficiently encoding the graph structure in PLMs is challenging because they were pretrained on natural language, and modeling structured data may lead to catastrophic forgetting of distributional knowledge. In this paper, we propose STRUC-TADAPT, an adapter method to encode graph structure into PLMs. Contrary to prior work, STRUCTADAPT effectively models interactions among the nodes based on the graph connectivity, only training graph structure-aware adapter parameters. In this way, we avoid catastrophic forgetting while maintaining the topological structure of the graph. We empirically show the benefits of explicitly encoding graph structure into PLMs using adapters and achieve state-of-the-art results on two AMRto-text datasets, training only 5.1% of the PLM parameters.

show abstract

Online Back-Parsing for AMR-to-Text Generation

Cited by 14 publications

References 32 publications

Promoting Graph Awareness in Linearized Graph-to-Text Generation

Promoting Graph Awareness in Linearized Graph-to-Text Generation

XLPT-AMR: Cross-Lingual Pre-Training via Multi-Task Learning for Zero-Shot AMR Parsing and Text Generation

Structural Adapters in Pretrained Language Models for AMR-to-text Generation

Contact Info

Product

Resources

About