Hierarchical Curriculum Learning for AMR Parsing

Abstract

Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, and is recently empowered by pretrained sequenceto-sequence models. However, there exists a gap between their flat training objective (i.e., equally treats all output tokens) and the hierarchical AMR structure, which limits the model generalization. To bridge this gap, we propose a Hierarchical Curriculum Learning (HCL) framework with Structure-level (SC) and Instance-level Curricula (IC). SC switches progressively from core to detail AMR semantic elements while IC transits from structuresimple to -complex AMR instances during training. Through these two warming-up processes, HCL reduces the difficulty of learning complex structures, thus the flat model can better adapt to the AMR hierarchy. Extensive experiments on AMR2.0, AMR3.0, structurecomplex and out-of-distribution situations verify the effectiveness of HCL. * Equal Contribution.

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs

“…AMR Parsing AMR parsing is a challenging semantic parsing task, since AMR is a deep semantic representation and consists of many separate annotations (Banarescu et al, 2013) (e.g., semantic relations, named entities, co-reference and so on). There are four major methods to do AMR Parsing currently, sequence-to-sequence approaches (Ge et al, 2019;Xu et al, 2020;Bevilacqua et al, 2021;Wang et al, 2022), tree-based approaches (Zhang et al, 2019b,a), graph-based approaches (Lyu and Titov, 2018;Cai and Lam, 2020) and transitionbased approaches (Naseem et al, 2019;Lee et al, 2020;Zhou et al, 2021a).…”

Addressing Long-Distance Dependencies in AMR Parsing with Hierarchical Clause Annotation

“…In seq2seq-based approaches, Bevilacqua et al [15] employ the Transformer-based pretrained language model, BART [16], to address LDDs in long sentences. Among these categories, seq2seq-based approaches have become mainstream, and recent parsers [17][18][19][20] employ the seq2seq architecture with the popular codebase SPRING [15], achieving better performance. Notably, HGAN [20] integrates token-level features, syntactic dependencies (SDP), and SRL with heterogeneous graph neural networks and has become the state-ofthe-art (SOTA) in terms of removing extra silver training data, graph-categorization, and ensemble methods.…”

“…However, these AMR parsers still suffer performance degradation when encountering long sentences with deeper AMR graphs [18,20] that introduce most LDD cases. We argue that the complexity of the clausal structure inside a sentence is the essence of LDDs, where clauses are the core units of grammar and center on a verb that determines the occurrences of other constituents [21].…”

“…Seq2seq-based: Model the task as transduction of the sentence into a linearization of the AMR graph [15,[17][18][19][20][35][36][37][38]. SPRING [15] is a popular seq2seq-based codebase that employs transfer learning by exploiting a pretrained encoder-decoder model, BART [16], to generate a linearized graph incrementally with a single auto-regressive pass of a seq2seq decoder.…”

“…SPRING [15] is a popular seq2seq-based codebase that employs transfer learning by exploiting a pretrained encoder-decoder model, BART [16], to generate a linearized graph incrementally with a single auto-regressive pass of a seq2seq decoder. The subsequent models, ANCES [17], HCL [18], ATP [19], and HGAN [20] all follow the architecture of SPRING, and HGAN integrates SDP and SRL features with heterogeneous graph neural networks to achieve the best performance in the tasks of removing extra silver training data, graph re-categorization, and ensemble methods.…”