Unifying Cross-Lingual Semantic Role Labeling with Heterogeneous Linguistic Resources

Abstract

While cross-lingual techniques are finding increasing success in a wide range of Natural Language Processing tasks, their application to Semantic Role Labeling (SRL) has been strongly limited by the fact that each language adopts its own linguistic formalism, from Prop-Bank for English to AnCora for Spanish and PDT-Vallex for Czech, inter alia. In this work, we address this issue and present a unified model to perform cross-lingual SRL over heterogeneous linguistic resources. Our model implicitly learns a high-quality mapping for different formalisms across diverse languages without resorting to word alignment and/or translation techniques. We find that, not only is our cross-lingual system competitive with the current state of the art but that it is also robust to low-data scenarios. Most interestingly, our unified model is able to annotate a sentence in a single forward pass with all the inventories it was trained with, providing a tool for the analysis and comparison of linguistic theories across different languages. We release our code and model at https://github.com/ SapienzaNLP/unify-srl.

Zero-shot Cross-lingual Conversational Semantic Role Labeling

“…Furthermore, we also find that fine-tuning all parameters leads to slightly better performance than freezing the language model during the CSRL training stage. This finding is consistent with the previous work (Conia et al, 2021). For the pre-training objectives, we found that (1) removing TLM & HPSI objective hurts the performance consistently but slightly; (2) SPI & UOR objectives help the model to better identify the cross-arguments; (3) SAI objective helps to find intra-arguments on DuConv and Persona-Chat, but might hurt the F1 intra score on CMU-DoG; (4) hierarchical pre-training is superior to end-to-end pre-training which simultaneously optimizes all auxiliary objectives.…”

“…Cross-lingual Language Model (CLM) We concatenate all utterances into a sequence and then use a pre-trained cross-lingual language model such as XLM-R (Conneau et al, 2020) or mBERT (Devlin et al, 2019) to capture the syntactic and semantic characteristics. Following Conia et al (2021), we obtain word representations e ∈ R |S|×d by concatenating the hidden states of the four top-most layers of the language model, where |S| is the sequence length and d is the dimension of the hidden state.…”

“…Recent works have witnessed prominent performances of multilingual pre-trained language models (PrLMs) (Devlin et al, 2019;Conneau et al, 2020) on cross-lingual tasks, including machine translation (Lin et al, 2020;Chen et al, 2021), semantic role labeling (SRL) (Conia and Navigli, 2020;Conia et al, 2021) and semantic parsing (Fei et al, 2020b;Sherborne and Lapata, 2021). How-ever, cross-lingual CSRL, as a combination of three challenging tasks (i.e., cross-lingual task, dialogue task and SRL task), suffers three outstanding difficulties: 1) latent space alignment -how to map word representations of different languages into an overlapping space; 2) conversation structure encoding -how to capture high-level dialogue features such as speaker dependency and temporal dependency; and 3) semantic arguments identification -how to highlight the relations between the predicate and its arguments, wherein PrLMs can only partially encode multilingual inputs to an overlapping vector space.…”

MuCPAD: A Multi-Domain Chinese Predicate-Argument Dataset

“…Specifically, we present a simple basic SRL model and enhance the model with the contextualized word representations from BERT for further improvements. Besides, we also present a MTL framework to improve the SRL performance by learning from multiple heterogeneous datasets simultaneously (Conia et al, 2021).…”

NER4ID at SemEval-2022 Task 2: Named Entity Recognition for Idiomaticity Detection

“…However, since idiomaticity is a frequent phenomenon that can be observed in all languages, idiomatic expressions should play an important role in NLP. Indeed, their identification and understanding is crucial not only for Natural Language Understanding tasks such as Word Sense Disambiguation (Bevilacqua et al, 2021b), Semantic Role Labeling (Conia et al, 2021) and Semantic Parsing (Bevilacqua et al, 2021a), but also for Machine Translation (Edunov et al, 2018;Liu et al, 2020), Question Answering (Mishra and Jain, 2016) and Text Summarization (Chu and Wang, 2018), inter alia.…”