Unsupervised Machine Translation Using Monolingual Corpora Only

Lample, Guillaume; Conneau, Alexis; Denoyer, Ludovic; Ranzato, Marc’Aurelio

doi:10.48550/arxiv.1711.00043

Cited by 93 publications

(132 citation statements)

References 24 publications

Supporting

Mentioning

130

Contrasting

Order By: Relevance

“…In recent years, especially with the success of transfer learning (Wang et al, 2018) and pretraining in NLP (Devlin et al, 2019), several techniques for improving neural MT for low-resource languages have been proposed (Sennrich et al, 2016;Fadaee et al, 2017;Xia et al, 2019;Lample et al, 2017;Lewis et al, 2019;Liu et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Don't Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data

Bhatnagar¹,

Ganesh²,

Kann³

2021

Preprint

View full text Add to dashboard Cite

High-performing machine translation (MT) systems can help overcome language barriers while making it possible for everyone to communicate and use language technologies in the language of their choice. However, such systems require large amounts of parallel sentences for training, and translators can be difficult to find and expensive. Here, we present a data collection strategy for MT which, in contrast, is cheap and simple, as it does not require bilingual speakers. Based on the insight that humans pay specific attention to movements, we use graphics interchange formats (GIFs) as a pivot to collect parallel sentences from monolingual annotators. We use our strategy to collect data in Hindi, Tamil and English. As a baseline, we also collect data using images as a pivot. We perform an intrinsic evaluation by manually evaluating a subset of the sentence pairs and an extrinsic evaluation by finetuning mBART (Liu et al., 2020) on the collected data. We find that sentences collected via GIFs are indeed of higher quality.

show abstract

Section: Related Workmentioning

confidence: 99%

Don't Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data

Bhatnagar¹,

Ganesh²,

Kann³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Because of the different task natures, balancing the training procedure with all the tasks is a critical problem for multi-task learning. Loss-balancing strategies (Chen et al, 2018b;Kendall et al, 2018;Liu et al, 2019;Gong et al, 2019;Guo et al, 2018a;Lample et al, 2017;Yao et al, 2019) are suitable for situations in which there are multiple training objectives that can be combined via weighted summation for each data point. In contrast, for multi-task learning across different datasets, a sampling strategy should be applied to decide the mixing ratio (how many batches to sample from each task) in each epoch.…”

Section: Multi-task Learningmentioning

confidence: 99%

MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Task

Yao

Papakostas

Burzo

et al. 2021

Preprint

View full text Add to dashboard Cite

The capability to automatically detect human stress can benefit artificial intelligent agents involved in affective computing and humancomputer interaction. Stress and emotion are both human affective states, and stress has proven to have important implications on the regulation and expression of emotion. Although a series of methods have been established for multimodal stress detection, limited steps have been taken to explore the underlying inter-dependence between stress and emotion. In this work, we investigate the value of emotion recognition as an auxiliary task to improve stress detection. We propose MUSER -a transformer-based model architecture and a novel multi-task learning algorithm with speed-based dynamic sampling strategy. Evaluations on the Multimodal Stressed Emotion (MuSE) dataset show that our model is effective for stress detection with both internal and external auxiliary tasks, and achieves state-ofthe-art results.

show abstract

“…However, in the Textcap task as well as many visionlanguage applications, such large-scale annotations are not readily available, and are both time-consuming and laborintensive to acquire. In these scenarios, unsupervised/unpaired methods (Lample et al 2017;Gu et al 2019;Caron et al 2020) that can learn vision to language or conversely from unpaired training data are highly desirable. Based on this insight, our goal is to break away from single and paired captioning conventions and conduct the unpaired captioning paradigm, which encourages TextCap models to generate diverse textual descriptions to comprehend images better.…”

Section: Introductionmentioning

confidence: 99%

MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning

Zhang¹,

Shi²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Text-based image captioning (TextCap) requires simultaneous comprehension of visual content and reading the text of images to generate a natural language description. Although a task can teach machines to understand the complex human environment further given that text is omnipresent in our daily surroundings, it poses additional challenges in normal captioning. A text-based image intuitively contains abundant and complex multimodal relational content, that is, image details can be described diversely from multiview rather than a single caption. Certainly, we can introduce additional paired training data to show the diversity of images' descriptions, this process is labor-intensive and time-consuming for TextCap pair annotations with extra texts. Based on the insight mentioned above, we investigate how to generate diverse captions that focus on different image parts using an unpaired training paradigm. We propose the Multimodal relAtional Graph adversarIal inferenCe (MAGIC) framework for diverse and unpaired TextCap. This framework can adaptively construct multiple multimodal relational graphs of images and model complex relationships among graphs to represent descriptive diversity. Moreover, a cascaded generative adversarial network is developed from modeled graphs to infer the unpaired caption generation in image-sentence feature alignment and linguistic coherence levels. We validate the effectiveness of MAGIC in generating diverse captions from different relational information items of an image. Experimental results show that MAGIC can generate very promising outcomes without using any image-caption training pairs.

show abstract

Unsupervised Machine Translation Using Monolingual Corpora Only

Cited by 93 publications

References 24 publications

Don't Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data

Don't Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data

MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Task

MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning

Contact Info

Product

Resources

About