The Meaning Factory at SemEval-2016 Task 8: Producing AMRs with Boxer

Bjerva, Johannes; Bos, Johannes; Haagsma, Hessel

doi:10.18653/v1/s16-1182

Cited by 12 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Grammar-based Parsing (CAMR) perform a series of shift-reduce transformations on the output of an externally-trained dependency parser, similar to Damonte et al (2017), Brandt et al (2016), Puzikov et al (2016), and Goodman et al (2016. Artzi et al (2015) use a grammar induction approach with Combinatory Categorical Grammar (CCG), which relies on pretrained CCGBank categories, like Bjerva et al (2016). Pust et al (2015) recast parsing as a string-to-tree Machine Translation problem, using unsupervised alignments (Pourdamghani et al, 2014), and employing several external semantic resources.…”

Section: Related Workmentioning

confidence: 99%

Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

Konstas¹,

Iyer²,

Yatskar³

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

244

406

View full text Add to dashboard Cite

Sequence-to-sequence models have shown strong performance across a broad range of applications. However, their application to parsing and generating text using Abstract Meaning Representation (AMR) has been limited, due to the relatively limited amount of labeled data and the nonsequential nature of the AMR graphs. We present a novel training procedure that can lift this limitation using millions of unlabeled sentences and careful preprocessing of the AMR graphs. For AMR parsing, our model achieves competitive results of 62.1 SMATCH, the current best score reported without significant use of external semantic resources. For AMR generation, our model establishes a new state-of-the-art performance of BLEU 33.8. We present extensive ablative and qualitative analysis including strong evidence that sequencebased AMR models are robust against ordering variations of graph-to-sequence conversions.

show abstract

Section: Related Workmentioning

confidence: 99%

Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

Konstas¹,

Iyer²,

Yatskar³

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

244

406

View full text Add to dashboard Cite

show abstract

“…This information is passed into an adapted Tarskian satisfaction relation for a Dynamic Semantics that is used to transform a syntactic parse into a predicate logic based meaning representation, followed by conversion to the required Penman notation. (Bjerva et al, 2016) This team employed an existing open-domain semantic parser, Boxer (Curran et al, 2007), which produces semantic representations based on Discourse Representation Theory. As the meaning representations produced by Boxer are considerably different from AMRs, the team used a hybrid conversion method to map Boxer's output to AMRs.…”

Section: Other Approachesmentioning

confidence: 99%

SemEval-2016 Task 8: Meaning Representation Parsing

May¹

2016

Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)

View full text Add to dashboard Cite

In this report we summarize the results of the SemEval 2016 Task 8: Meaning Representation Parsing. Participants were asked to generate Abstract Meaning Representation (AMR) (Banarescu et al., 2013) graphs for a set of English sentences in the news and discussion forum domains. Eleven sites submitted valid systems. The availability of state-of-the-art baseline systems was a key factor in lowering the bar to entry; many submissions relied on CAMR (Wang et al., 2015b; Wang et al., 2015a) as a baseline system and added extensions to it to improve scores. The evaluation set was quite difficult to parse, particularly due to creative approaches to word representation in the web forum portion. The top scoring systems scored 0.62 F1 according to the Smatch (Cai and Knight, 2013) evaluation heuristic. We show some sample sentences along with a comparison of system parses and perform quantitative ablative studies.

show abstract

“…Training For the optimization of the accuracy prediction model we use only the development and training sections of LDC2015E86 and the corresponding automatic parses together with the gold scores. Details on the training cycle can be found in the Supplemental Material §A (the loss is de-DynamicPower (Butler, 2016), TMF (Bjerva et al, 2016), UCL+Sheffield (Goodman et al, 2016) and CU-NLP (Foland and Martin, 2016). 7 TMF-1 and TMF-2 (van Noord and Bos, 2017a), DAN-GNT (Nguyen and Nguyen, 2017), Oxford (Buys and Blunsom, 2017), RIGOTRIO (Gruzitis et al, 2017) and JAMR (Flanigan et al, 2016) 8 https://spacy.io/ scribed in §4).…”

Section: Methodsmentioning

confidence: 99%

Automatic Accuracy Prediction for AMR Parsing

Opitz

Frank

2019

Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)

View full text Add to dashboard Cite

Meaning Representation (AMR) represents sentences as directed, acyclic and rooted graphs, aiming at capturing their meaning in a machine readable format. AMR parsing converts natural language sentences into such graphs. However, evaluating a parser on new data by means of comparison to manually created AMR graphs is very costly. Also, we would like to be able to detect parses of questionable quality, or preferring results of alternative systems by selecting the ones for which we can assess good quality. We propose AMR accuracy prediction as the task of predicting several metrics of correctness for an automatically generated AMR parse -in absence of the corresponding gold parse. We develop a neural end-to-end multi-output regression model and perform three case studies: firstly, we evaluate the model's capacity of predicting AMR parse accuracies and test whether it can reliably assign high scores to gold parses. Secondly, we perform parse selection based on predicted parse accuracies of candidate parses from alternative systems, with the aim of improving overall results. Finally, we predict system ranks for submissions from two AMR shared tasks on the basis of their predicted parse accuracy averages. All experiments are carried out across two different domains and show that our method is effective.

show abstract

The Meaning Factory at SemEval-2016 Task 8: Producing AMRs with Boxer

Cited by 12 publications

References 9 publications

Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

SemEval-2016 Task 8: Meaning Representation Parsing

Automatic Accuracy Prediction for AMR Parsing

Contact Info

Product

Resources

About