Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling

Krishnan, Jitin; Anastasopoulos, Antonios; Purohit, Hemant; Rangwala, Huzefa

doi:10.18653/v1/2021.mrl-1.18

Cited by 11 publications

(5 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When porting a dialogue system to new languages, zero-shot transfer is an effective method to bypass costly data collection and annotation for every target language. However, based on prior work in general-purpose cross-lingual NLP, we detect three crucial gaps which require more attention in future work and that may play an instrumental role in the final task performance: 1) the choice of source language(s) (Zoph et al, 2016;Dabre et al, 2017;Lin et al, 2019), as recently hinted at for multilingual ToD NLU by Krishnan et al (2021); 2) harnessing multiple source languages rather than a single one (Zoph et al, 2016;Pan et al, 2020;Wu et al, 2020b;Ansell et al, 2021); 3) few-shot transfer with a small number of target-language examples, as opposed to fully zero-shot transfer (Lauscher et al, 2020). In other words, the go-to option of always transferring from English in a zero-shot fashion might be sub-optimal for a large number of target languages.…”

Section: Coping With Low-resource Scenarios In Nlumentioning

confidence: 92%

“…Cross-Lingual Transfer Methods. Given the absence of sufficiently large training data in many languages, the default approach to multilingual NLU is (zero-shot or few-shot) transfer of models trained on English datasets by means of massively multilingual Transformer-based encoders (Zhang et al, 2019b;Xu et al, 2020;Siddhant et al, 2020b;Krishnan et al, 2021). While most of the work relies on MEs pretrained via masked language modelling, such as mBERT (Devlin et al, 2019) and XLM-R , Siddhant et al (2020b) show that MEs pretrained via machine translation leads to more effective zero-shot transfer for intent classification.…”

Section: Natural Language Understanding (Nlu)mentioning

confidence: 99%

“…Firstly, several automated approaches to reduce human effort were proposed, predominantly focusing on data augmentation. These approaches, previously applied to multilingual dialogue, include (i) word or span substitution, creating code-switched data between source and target languages Krishnan et al, 2021;Qin et al, 2021) or synonymous span substitution in a target language (Louvan & Magnini, 2020b)2020; and (ii) semi-supervised training with target language sentence retrieval (Razumovskaia et al, 2022).…”

Section: Outlook For Multilingual Tod Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Razumovskaia

Glavaš

Majewska

et al. 2022

jair

View full text Add to dashboard Cite

In task-oriented dialogue (ToD), a user holds a conversation with an artificial agent with the aim of completing a concrete task. Although this technology represents one of the central objectives of AI and has been the focus of ever more intense research and development efforts, it is currently limited to a few narrow domains (e.g., food ordering, ticket booking) and a handful of languages (e.g., English, Chinese). This work provides an extensive overview of existing methods and resources in multilingual ToD as an entry point to this exciting and emerging field. We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation. In fact, acquiring annotations or human feedback for each component of modular systems or for data-hungry end-to-end systems is expensive and tedious. Hence, state-of-the-art approaches to multilingual ToD mostly rely on (zero- or few-shot) cross-lingual transfer from resource-rich languages (almost exclusively English), either by means of (i) machine translation or (ii) multilingual representations. These approaches are currently viable only for typologically similar languages and languages with parallel / monolingual corpora available. On the other hand, their effectiveness beyond these boundaries is doubtful or hard to assess due to the lack of linguistically diverse benchmarks (especially for natural language generation and end-to-end evaluation). To overcome this limitation, we draw parallels between components of the ToD pipeline and other NLP tasks, which can inspire solutions for learning in low-resource scenarios. Finally, we list additional challenges that multilinguality poses for related areas (such as speech, fluency in generated text, and human-centred evaluation), and indicate future directions that hold promise to further expand language coverage and dialogue capabilities of current ToD systems.

show abstract

Section: Coping With Low-resource Scenarios In Nlumentioning

confidence: 92%

Section: Natural Language Understanding (Nlu)mentioning

confidence: 99%

Section: Outlook For Multilingual Tod Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Razumovskaia

Glavaš

Majewska

et al. 2022

jair

View full text Add to dashboard Cite

show abstract

“…Generally, these approaches additionally require word alignment to project annotations between languages. Prior zero-shot cross-lingual work in SLU (Li et al, 2021;Zhu et al, 2020;Krishnan et al, 2021) similarly identifies a penalty for cross-lingual transfer and finds that pre-trained models and machine translation can only partially mitigate this error.…”

Section: Related Workmentioning

confidence: 99%

Zero-Shot Cross-lingual Semantic Parsing

Sherborne¹,

Lapata²

2021

Preprint

View full text Add to dashboard Cite

Recent work in crosslingual semantic parsing has successfully applied machine translation to localize accurate parsing to new languages. However, these advances assume access to high-quality machine translation systems, and tools such as word aligners, for all test languages. We remove these assumptions and study cross-lingual semantic parsing as a zeroshot problem without parallel data for 7 test languages (DE, ZH, FR, ES, PT, HI, TR). We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-Logical form paired data and unlabeled, monolingual utterances in each test language. We train an encoder to generate language-agnostic representations jointly optimized for generating logical forms or utterance reconstruction and against language discriminability. Our system frames zero-shot parsing as a latent-space alignment problem and finds that pre-trained models can be improved to generate logical forms with minimal cross-lingual transfer penalty. Experimental results on Overnight and a new executable version of MultiATIS++ find that our zero-shot approach performs above backtranslation baselines and, in some cases, approaches the supervised upper bound.

show abstract

“…The progress in multilingual TOD is critically hampered by the paucity of training data for many of the world's languages. While cross-lingual transfer learning (Zhang et al, 2019;Xu et al, 2020;Krishnan et al, 2021) offers a partial remedy, its success is tenuous beyond typologically similar languages and generally hard to assess due to the lack of evaluation benchmarks (Razumovskaia et al, 2021). What is more, transfer learning often cannot leverage multi-source transfer and few-shot learning due to lack of language diversity in the TOD datasets (Zhu et al, 2020;Quan et al, 2020;Farajian et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation

Majewska¹,

Razumovskaia²,

Ponti³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multilingual task-oriented dialogue (TOD) facilitates access to services and information for many (communities of) speakers. Nevertheless, the potential of this technology is not fully realised, as current datasets for multilingual TOD-both for modular and end-toend modelling-suffer from severe limitations. 1) When created from scratch, they are usually small in scale and fail to cover many possible dialogue flows. 2) Translation-based TOD datasets might lack naturalness and cultural specificity in the target language. In this work, to tackle these limitations we propose a novel outline-based annotation process for multilingual TOD datasets, where domainspecific abstract schemata of dialogue are mapped into natural language outlines. These in turn guide the target language annotators in writing a dialogue by providing instructions about each turn's intents and slots. Through this process we annotate a new large-scale dataset for training and evaluation of multilingual and cross-lingual TOD systems. Our Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding, dialogue state tracking, and end-toend dialogue modelling and evaluation in 4 diverse languages: Arabic, Indonesian, Russian, and Kiswahili. Qualitative and quantitative analyses of COD versus an equivalent translation-based dataset demonstrate improvements in data quality, unlocked by the outline-based approach. Finally, we benchmark a series of state-of-the-art systems for cross-lingual TOD, setting reference scores for future work and demonstrating that COD prevents over-inflated performance, typically met with prior translation-based TOD datasets.

show abstract

Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling

Cited by 11 publications

References 61 publications

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Zero-Shot Cross-lingual Semantic Parsing

Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation

Contact Info

Product

Resources

About