Adversarial Training for Cross-Domain Universal Dependency Parsing

Abstract

We describe our submission to the CoNLL 2017 shared task, which exploits the shared common knowledge of a language across different domains via a domain adaptation technique. Our approach is an extension to the recently proposed adversarial training technique for domain adaptation, which we apply on top of a graph-based neural dependency parsing model on bidirectional LSTMs. In our experiments, we find our baseline graphbased parser already outperforms the official baseline model (UDPipe) by a large margin. Further, by applying our technique to the treebanks of the same language with different domains, we observe an additional gain in the performance, in particular for the domains with less training data.

Parser Training with Heterogeneous Treebanks

“…When the data is very skewed, as for Russian, the effect of adding a small treebank to a large one is minor, as expected. While our results are not directly comparable to the adversarial learning in Sato et al (2017) who used a different parser and test set, the improvements of C+FT and TB-EMB are typically at least on par with and often larger than their improvements. While our im- For each test set, the best result is marked with bold.…”

Neural Unsupervised Domain Adaptation in NLP—A Survey

“…DANNs have been applied in many NLP tasks in the last few years, mainly to sentiment classification (e.g., Ganin et al (2016), Li et al (2018a), Shen et al (2018), Rocha andLopes Cardoso (2019), Ghoshal et al (2020), to name a few), but recently to many other tasks as well: language identification (Li et al, 2018a), natural language inference (Rocha and Lopes Cardoso, 2019), POS tagging (Yasunaga et al, 2018), parsing (Sato et al, 2017), trigger identification (Naik and Rose, 2020), relation extraction Fu et al, 2017;Rios et al, 2018), and other (binary) text classification tasks like relevancy identification (Alam et al, 2018a), machine reading comprehension , stance detection (Xu et al, 2019), and duplicate question detection (Shah et al, 2018). This makes DANNs the most widely used UDA approach in NLP, as illustrated in Table 1.…”

A Simple yet Effective Joint Training Method for Cross-Lingual Universal Dependency Parsing

“…where w l indicates lexical feature, w d indicates delexicalized feature, and is element-wise multiplication. The difference between Sato et al (2017) and ours is that we remove the adversarial training loss, which is because we have already use the universal information in the shared network.…”

“…Beyond embedding-based methods, a natural question is whether we can use a simple way to utilize the universal information. Some previous research either regarded the universal information as extra training signals (e.g., delexicalized embedding (Dehouck and Denis, 2017)), or implicitly trained a network with all features (e.g., adversarial training for parsing in Sato et al (2017)). In our system, we manually and explicitly share the universal annotations via a shared LSTM component.…”