PPT: Parsimonious Parser Transfer for Unsupervised Cross-Lingual Adaptation

Kurniawan, Kemal; Frermann, Lea; Schulz, Philip; Cohn, Trevor

doi:10.18653/v1/2021.eacl-main.254

Cited by 7 publications

(13 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multilinguality is the key factor contributing to the success of PPTX (Kurniawan et al, 2021). Therefore, optimising the method to leverage this multilinguality provided by the source models is important.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Model Architecture For parsing, we use the same architecture as was used by Kurniawan et al (2021), consisting of embedding layers, a Transformer encoder layer, and a biaffine output layer (Dozat and Manning, 2017). At test time, we run the MST algorithm (Chu and Liu, 1965;Edmonds, 1967) to find the highest scoring tree.…”

Section: Methodsmentioning

confidence: 99%

“…Source Selection We adopt a "pragmatic" approach where we include 5 high-resource languages as sources: English, Arabic, Spanish, French, and German (Kurniawan et al, 2021), 4 which have been categorised as "quintessential richresource languages" due to the availability of massive language datasets (Joshi et al, 2020). When a source language is also the target language, we exclude the language from the sources.…”

Section: Methodsmentioning

confidence: 99%

“…In multi-source transfer, the set Ỹ(x) can be obtained by an ensemble method applied to the source models. PPTX (Kurniawan et al, 2021) is one such method designed for arc-factored dependency parsers. We generalise PPTX, making it applicable to any set of source models that predict structured outputs that decompose into substructures (of which a set of arc-factored dependency parsers is a special case).…”

Section: Supervision Via Ensemblementioning

confidence: 99%

“…One recent method for unsupervised crosslingual transfer is PPTX (Kurniawan et al, 2021). Developed for dependency parsing, it transfers from multiple source languages, which has been shown to be superior to transferring from just a single language (McDonald et al, 2011;Duong et al, 2015;Rahimi et al, 2019, inter alia).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Unsupervised Cross-Lingual Transfer of Structured Predictors without Source Data

Kurniawan¹,

Frermann²,

Schulz³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

Providing technologies to communities or domains where training data is scarce or protected e.g., for privacy reasons, is becoming increasingly important. To that end, we generalise methods for unsupervised transfer from multiple input models for structured prediction. We show that the means of aggregating over the input models is critical, and that multiplying marginal probabilities of substructures to obtain high-probability structures for distant supervision is substantially better than taking the union of such structures over the input models, as done in prior work. Testing on 18 languages, we demonstrate that the method works in a cross-lingual setting, considering both dependency parsing and part-of-speech structured prediction problems. Our analyses show that the proposed method produces less noisy labels for the distant supervision. 1

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Supervision Via Ensemblementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Unsupervised Cross-Lingual Transfer of Structured Predictors without Source Data

Kurniawan¹,

Frermann²,

Schulz³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

show abstract

Cross-Domain Transfer Learning for Dependency Parsing

Zhou

Zhao

et al. 2019

Natural Language Processing and Chinese Computing

View full text Add to dashboard Cite

While structure learning achieves remarkable performance in high-resource languages, the situation differs for under-represented languages due to the scarcity of annotated data. This study focuses on assessing the efficacy of transfer learning in enhancing dependency parsing for Javanese-a language spoken by 80 million individuals but characterized by limited representation in natural language processing. We utilized the Universal Dependencies dataset consisting of dependency treebanks from more than 100 languages, including Javanese. We propose two learning strategies to train the model: transfer learning (TL) and hierarchical transfer learning (HTL). While TL only uses a source language to pre-train the model, the HTL method uses a source language and an intermediate language in the learning process. The results show that our best model uses the HTL method, which improves performance with an increase of 10 % for both UAS and LAS evaluations compared to the baseline model.

show abstract