ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations

Wieting, John; Gimpel, Kevin

doi:10.18653/v1/p18-1042

Cited by 266 publications

(221 citation statements)

References 48 publications

Supporting

Mentioning

217

Contrasting

Unclassified

Order By: Relevance

“…This motivates the case for exploiting incidental annotation (Roth, 2017) and automating some aspects of dataset creation. The current trend of using machine translation systems to produce augmented datasets for machine translation itself (Sennrich et al, 2016) and for monolingual tasks like classification (Yu et al, 2018) and paraphrasing (Wieting and Gimpel, 2018) is a good example of this.…”

Section: Datamentioning

confidence: 99%

Large-Scale Representation Learning from Visually Grounded Untranscribed Speech

Ilharco

Zhang

Baldridge

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

Systems that can associate images with their spoken audio captions are an important step towards visually grounded language learning. We describe a scalable method to automatically generate diverse audio for image captioning datasets. This supports pretraining deep networks for encoding both audio and images, which we do via a dual encoder that learns to align latent representations from both modalities. We show that a masked margin softmax loss for such models is superior to the standard triplet loss. We fine-tune these models on the Flickr8k Audio Captions Corpus and obtain state-of-the-art results-improving recall in the top 10 from 29.6% to 49.5%. We also obtain human ratings on retrieval outputs to better assess the impact of incidentally matching image-caption pairs that were not associated in the data, finding that automatic evaluation substantially underestimates the quality of the retrieved results. * Work done as a member of the Google AI Residency Program.

show abstract

Section: Datamentioning

confidence: 99%

Large-Scale Representation Learning from Visually Grounded Untranscribed Speech

Ilharco

Zhang

Baldridge

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

show abstract

“…As our bitext, we use the Czeng1.6 English-Czech parallel corpus (Bojar et al, 2016). We compare it to training on ParaNMT (Wieting and Gimpel, 2018), a corpus of 50 million paraphrases obtained from automatically translating the Czech side of Czeng1.6 into English. We sample 1 million examples from ParaNMT and Czeng1.6 and evaluate on all 25 datasets from the STS tasks from 2012-2016.…”

Section: Back-translated Text Vs Parallel Textmentioning

confidence: 99%

Simple and Effective Paraphrastic Similarity from Parallel Translations

Wieting¹,

Gimpel²,

Neubig³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The model computes the average of vectors of all words and n-grams in a sentence. Several studies have shown successful results produced by Sent2vec [32,33]. The model learns a context embedding v w and target embedding u w for each word w in the vocabulary, with h number of embedding dimensions.…”

Section: Sent2vecmentioning

confidence: 99%

Deep Contextualized Pairwise Semantic Similarity for Arabic Language Questions

Al-Bataineh

Farhan

Mustafa

et al. 2019

2019 IEEE 31st International Conference on Tools With Artificial Intelligence (ICTAI)

View full text Add to dashboard Cite

Question semantic similarity is a challenging and active research problem that is very useful in many NLP applications, such as detecting duplicate questions in community question answering platforms such as Quora. Arabic is considered to be an under-resourced language, has many dialects, and rich in morphology. Combined together, these challenges make identifying semantically similar questions in Arabic even more difficult. In this paper, we introduce a novel approach to tackle this problem, and test it on two benchmarks; one for Modern Standard Arabic (MSA), and another for the 24 major Arabic dialects. We are able to show that our new system outperforms state-of-the-art approaches by achieving 93% F1-score on the MSA benchmark and 82% on the dialectical one. This is achieved by utilizing contextualized word representations (ELMo embeddings) trained on a text corpus containing MSA and dialectic sentences. This in combination with a pairwise fine-grained similarity layer, helps our question-to-question similarity model to generalize predictions on different dialects while being trained only on question-to-question MSA data.

show abstract

ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations

Cited by 266 publications

References 48 publications

Large-Scale Representation Learning from Visually Grounded Untranscribed Speech

Large-Scale Representation Learning from Visually Grounded Untranscribed Speech

Simple and Effective Paraphrastic Similarity from Parallel Translations

Deep Contextualized Pairwise Semantic Similarity for Arabic Language Questions

Contact Info

Product

Resources

About