Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification

Zhang, Mozhi; Fujinuma, Yoshinari; Boyd-Graber, Jordan

doi:10.1609/aaai.v34i05.6500

Cited by 13 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Overall, the best method to generalize the generation model across languages is to use machine-translated data. (Ammar et al, 2016;Cotterell and Heigold, 2017;Ahmad et al, 2019;Lin et al, 2019;Zhang et al, 2020a). We are also interested in training on both organic data and MT data; i.e., mixing the zero-shot and MT setting.…”

Section: Results On Other Languagesmentioning

confidence: 99%

A Dataset and Baselines for Multilingual Reply Suggestion

Zhang¹,

Wang²,

Deb³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

Reply suggestion models help users process emails and chats faster. Previous work only studies English reply suggestion. Instead, we present MRS, a multilingual reply suggestion dataset with ten languages. MRS can be used to compare two families of models: 1) retrieval models that select the reply from a fixed set and 2) generation models that produce the reply from scratch. Therefore, MRS complements existing cross-lingual generalization benchmarks that focus on classification and sequence labeling tasks. We build a generation model and a retrieval model as baselines for MRS. The two models have different strengths in the monolingual setting, and they require different strategies to generalize across languages. MRS is publicly available at https://github.com/zhangmozhi/mrs. Multilingual Reply SuggestionAutomated reply suggestion is a useful feature for email and chat applications. Given an input message, the system suggests several replies, and users may click on them to save typing time (Figure 1). This feature is available in many applications including Gmail, Outlook, LinkedIn, Facebook Messenger, Microsoft Teams, and Uber.Reply suggestion is related to but different from open-domain dialog systems or chatbots (Adiwardana et al., 2020;. While both are conversational AI tasks (Gao et al., 2019), the goals are different: reply suggestion systems help the user quickly reply to a message, while chatbots aim to continue the conversation and focus more on multi-turn dialogues.Ideally, we want our model to generate replies in any language. However, reply suggestion models require large training sets, so previous work mostly * Work mostly done as an intern at Microsoft Research.

show abstract

Section: Results On Other Languagesmentioning

confidence: 99%

A Dataset and Baselines for Multilingual Reply Suggestion

Zhang¹,

Wang²,

Deb³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, CLTC assumes that source labeled data are available, and the source and target tasks are identical (Upadhyay et al, 2016;Karamanolakis et al, 2020;Xu et al, 2016;Bel et al, 2003). Since the data requirement of CLTC can be restrictive, there exist recent methods for weakly supervised CLTC where target labels are not required (Karamanolakis et al, 2020;Xu et al, 2016;Zhang et al, 2020a). Note that source labeled data are still required for such methods.…”

Section: Related Workmentioning

confidence: 99%

Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph

Chairatanakul

Sriwatanasakdi²,

Charoenphakdee

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

In cross-lingual text classification, it is required that task-specific training data in high-resource source languages are available, where the task is identical to that of a low-resource target language. However, collecting such training data can be infeasible because of the labeling cost, task characteristics, and privacy concerns. This paper proposes an alternative solution that uses only task-independent word embeddings of high-resource languages and bilingual dictionaries. First, we construct a dictionary-based heterogeneous graph (DHG) from bilingual dictionaries. This opens the possibility to use graph neural networks for crosslingual transfer. The remaining challenge is the heterogeneity of DHG because multiple languages are considered. To address this challenge, we propose dictionary-based heterogeneous graph neural network (DHGNet) that effectively handles the heterogeneity of DHG by two-step aggregations, which are word-level and language-level aggregations. Experimental results demonstrate that our method outperforms pretrained models even though it does not access to large corpora. Furthermore, it can perform well even though dictionaries contain many incorrect translations. Its robustness allows the usage of a wider range of dictionaries such as an automatically constructed dictionary and crowdsourced dictionary, which are convenient for real-world applications.

show abstract

“…Cross-Lingual Document Classification. Prior approaches transfer knowledge with cross-lingual resources, such as bilingual dictionaries (Wu et al, 2008;Shi et al, 2010), parallel text (Xu and Yang, 2017), labeled data from related languages (Zhang et al, 2020a), structural correspondences (Peter Prettenhofer, 2010), multilingual topic models (Ni et al, 2011;Andrade et al, 2015), machine translation (Wan, 2009;Zhou et al, 2016), and CLWE (Klementiev et al, 2012). Our method instead brings a bilingual speaker in the loop to actively provide cross-lingual knowledge, which is more reliable in low-resource settings.…”

Section: Related Workmentioning

confidence: 99%

“…These climes help sentiment analysis. Zhang et al, 2020a). We develop CLassifying Interactively with Multilingual Embeddings (CLIME), that efficiently specializes CLWE with human interaction.…”

Section: Introductionmentioning

confidence: 99%

Interactive Refinement of Cross-Lingual Word Embeddings

Yuan

Zhang

Durme

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

Cross-lingual word embeddings transfer knowledge between languages: models trained on high-resource languages can predict in low-resource languages. We introduce CLIME, an interactive system to quickly refine cross-lingual word embeddings for a given classification problem. First, CLIME ranks words by their salience to the downstream task. Then, users mark similarity between keywords and their nearest neighbors in the embedding space. Finally, CLIME updates the embeddings using the annotations. We evaluate CLIME on identifying health-related text in four low-resource languages: Ilocano, Sinhalese, Tigrinya, and Uyghur. Embeddings refined by CLIME capture more nuanced word semantics and have higher test accuracy than the original embeddings. CLIME often improves accuracy faster than an active learning baseline and can be easily combined with active learning to improve results.

show abstract

Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification

Cited by 13 publications

References 27 publications

A Dataset and Baselines for Multilingual Reply Suggestion

A Dataset and Baselines for Multilingual Reply Suggestion

Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph

Interactive Refinement of Cross-Lingual Word Embeddings

Contact Info

Product

Resources

About