One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets

Damonte, Marco; Monti, Emilio

doi:10.18653/v1/2021.starsem-1.16

Cited by 3 publications

(4 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) How to choose auxiliary task? The task selection is important since loosely related tasks may even impede the AMR parsing according to Damonte and Monti (2021). However, in literature there are no principles or consensus on how to choose the proper auxiliary tasks for AMR parsing.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Machine Translation generates text sequence while Dependency Parsing (DP) and Semantic Role Labeling (SRL) produces dependency trees and semantic role forests respectively as shown in Figure 1. Prior studies (Xu et al, 2020;Wu et al, 2021;Damonte and Monti, 2021) do not attach particular importance to the gap, which might lead the auxiliary tasks to outlier-task Cai et al, 2017) in the Multitask Learning, deteriorating the performance of AMR parsing. 3) How to introduce auxiliary tasks more effectively?…”

Section: Introductionmentioning

confidence: 99%

“…3) How to introduce auxiliary tasks more effectively? After investigating different training paradigms to combine the auxiliary task training with the major objective (AMR parsing), we figure out that, although all baseline models (Xu et al, 2020;Wu et al, 2021;Damonte and Monti, 2021) choose to jointly train the auxiliary tasks and AMR parsing with Multitask Learning (MTL), Intermediate-task Learning (ITL) is a more effective way to introduce the auxiliary tasks for pretrained models. Our observation is also consistent with (Pruksachatkun et al, 2020;Poth et al, 2021), which improve other NLP tasks with enhanced pretrained models.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs

Chen¹,

Wang²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Meaning Representation (AMR) implicitly involves compound semantic annotations, we hypothesize auxiliary tasks which are semantically or formally related can better enhance AMR parsing. We find that 1) Semantic role labeling (SRL) and dependency parsing (DP), would bring more performance gain than other tasks e.g. MT and summarization in the text-to-AMR transition even with much less data. 2) To make a better fit for AMR, data from auxiliary tasks should be properly "AMRized" to PseudoAMR before training. Knowledge from shallow level parsing tasks can be better transferred to AMR Parsing with structure transform. 3) Intermediatetask learning is a better paradigm to introduce auxiliary tasks to AMR parsing, compared to multitask learning. From an empirical perspective, we propose a principled method to involve auxiliary tasks to boost AMR parsing. Extensive experiments show that our method achieves new state-of-the-art performance on different benchmarks especially in topology-related scores. Code and models are released at https://github. com/PKUnlp-icler/ATP.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs

Chen¹,

Wang²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…An example of this category would be a test that investigates how well one pretrained model generalises with respect to an o.o.d. finetuning train-test split(Damonte and Monti, 2021;Kavumba et al, 2022;Ludwig et al, 2022). The parts of the modelling pipeline that studies with a finetune train-test locus can evaluate are the same as studies with a train-test locus, although studies that investigate the generalisation abilities of a single finetuned model instance are rare.…”

mentioning

confidence: 99%

State-of-the-art generalisation research in NLP: A taxonomy and review

Hupkes¹,

Giulianelli²,

Dankers³

et al. 2022

Preprint

View full text Add to dashboard Cite

The ability to generalise well is one of the primary desiderata of natural language processing (NLP). Yet, what 'good generalisation' entails and how it should be evaluated is not well understood, nor are there any common standards to evaluate it. In this paper, we aim to lay the groundwork to improve both of these issues. We present a taxonomy for characterising and understanding generalisation research in NLP, we use that taxonomy to present a comprehensive map of published generalisation studies, and we make recommendations for which areas might deserve attention in the future. Our taxonomy is based on an extensive literature review of generalisation research, and contains five axes along which studies can differ: their main motivation, the type of generalisation they aim to solve, the type of data shift they consider, the source by which this data shift is obtained, and the locus of the shift within the modelling pipeline. We use our taxonomy to classify over 400 previous papers that test generalisation, for a total of more than 600 individual experiments. Considering the results of this review, we present an in-depth analysis of the current state of generalisation research in NLP, and make recommendations for the future. Along with this paper, we release a webpage where the results of our review can be dynamically explored, and which we intend to update as new NLP generalisation studies are published. With this work, we aim to make steps towards making state-of-the-art generalisation testing the new status quo in NLP.

show abstract

An Analysis of Data Processing for Big Data Analytics

Blair

Cotter

2021

JCNS

View full text Add to dashboard Cite

The need for high-performance Data Mining (DM) algorithms is being driven by the exponentially increasing data availability such as images, audio and video from a variety of domains, including social networks and the Internet of Things (IoT). Deep learning is an emerging field of pattern recognition and Machine Learning (ML) study right now. It offers computer simulations of numerous nonlinear processing layers of neurons that may be used to learn and interpret data at higher degrees of abstractions. Deep learning models, which may be used in cloud technology and huge computational systems, can inherently capture complex structures of large data sets. Heterogeneousness is one of the most prominent characteristics of large data sets, and Heterogeneous Computing (HC) causes issues with system integration and Advanced Analytics. This article presents HC processing techniques, Big Data Analytics (BDA), large dataset instruments, and some classic ML and DM methodologies. The use of deep learning to Data Analytics is investigated. The benefits of integrating BDA, deep learning, HPC (High Performance Computing), and HC are highlighted. Data Analytics and coping with a wide range of data are discussed.

show abstract

One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets

Cited by 3 publications

References 45 publications

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs

State-of-the-art generalisation research in NLP: A taxonomy and review

An Analysis of Data Processing for Big Data Analytics

Contact Info

Product

Resources

About