Charagram: Embedding Words and Sentences via Character n-grams

Wieting, John; Bansal, Mohit; Gimpel, Kevin; Livescu, Karen

doi:10.18653/v1/d16-1157

Cited by 141 publications

(121 citation statements)

References 40 publications

(41 reference statements)

Supporting

Mentioning

121

Contrasting

Order By: Relevance

“…These representations are robust to out-of-vocabulary items, often producing improved results. Our pretraining procedure is reminiscent of several recent papers (Wieting et al, 2016, inter alia) who aim for general purpose character n-gram embeddings. In contrast, we pretrain all model parameters on automatic but in-domain paraphrase data.…”

Section: Related Workmentioning

confidence: 99%

Neural Paraphrase Identification of Questions with Noisy Pretraining

Tomar¹,

Duque²,

Täckström³

et al. 2017

Proceedings of the First Workshop on Subword and Character Level Models in NLP

View full text Add to dashboard Cite

We present a solution to the problem of paraphrase identification of questions. We focus on a recent dataset of question pairs annotated with binary paraphrase labels and show that a variant of the decomposable attention model (Parikh et al., 2016) results in accurate performance on this task, while being far simpler than many competing neural architectures. Furthermore, when the model is pretrained on a noisy dataset of automatically collected question paraphrases, it obtains the best reported performance on the dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Neural Paraphrase Identification of Questions with Noisy Pretraining

Tomar¹,

Duque²,

Täckström³

et al. 2017

Proceedings of the First Workshop on Subword and Character Level Models in NLP

View full text Add to dashboard Cite

show abstract

“…Other ways of representing words with embeddings have to be experimented with, especially the ones where word and character-level representations are mixed, like Charagram (Wieting et al, 2016). It is also interesting to see, whether this metric can be used for hill-climbing and system development.…”

Section: Discussionmentioning

confidence: 99%

bleu2vec: the Painfully Familiar Metric on Continuous Vector Space Steroids

Tättar¹,

Fishel²

2017

Proceedings of the Second Conference on Machine Translation

View full text Add to dashboard Cite

show abstract

“…There is a large literature on exploiting characters, morphology, and composition for embedding models (Chen et al, 2015;Ling et al, 2015a;Qiu et al, 2014;Wieting et al, 2016;Lazaridou et al, 2013), and a comparison with these different models may be interesting future work.…”

Section: Morphological Word Embeddingsmentioning

confidence: 99%

Proceedings of the Second Workshop on Subword/Character LEvel Models

2018

View full text Add to dashboard Cite

ii Introduction Traditional NLP starts with a hand-engineered layer of representation, the level of tokens or words. A tokenization component first breaks up the text into units using manually designed rules. Tokens are then processed by components such as word segmentation, morphological analysis and multiword recognition. The heterogeneity of these components makes it hard to create integrated models of both structure within tokens (e.g., morphology) and structure across multiple tokens (e.g., multi-word expressions). This approach can perform poorly (i) for morphologically rich languages, (ii) for noisy text, (iii) for languages in which the recognition of words is difficult and (iv) for adaptation to new domains; and (v) it can impede the optimization of preprocessing in end-to-end learning.The workshop provides a forum for discussing recent advances as well as future directions on sub-word and character-level natural language processing and representation learning that address these problems. Topics of Interest:• tokenization-free models• character-level machine translation• character-ngram information retrieval• transfer learning for character-level models• models of within-token and cross-token structure• NL generation (of words not seen in training etc)• out of vocabulary words• morphology and segmentation• relationship b/w morphology and character-level models• stemming and lemmatization • inflection generation• orthographic productivity• form-meaning representations• true end-to-end learning AbstractNeural machine translation has achieved impressive results in the last few years, but its success has been limited to settings with large amounts of parallel data. One way to improve NMT for lower-resource settings is to initialize a word-based NMT model with pretrained word embeddings. However, rare words still suffer from lower quality word embeddings when trained with standard word-level objectives. We introduce word embeddings that utilize morphological resources, and compare to purely unsupervised alternatives. We work with Arabic, a morphologically rich language with available linguistic resources, and perform Ar-to-En MT experiments on a small corpus of TED subtitles. We find that word embeddings utilizing subword information consistently outperform standard word embeddings on a word similarity task and as initialization of the source word embeddings in a low-resource NMT system.

show abstract

Charagram: Embedding Words and Sentences via Character n-grams

Cited by 141 publications

References 40 publications

Neural Paraphrase Identification of Questions with Noisy Pretraining

Neural Paraphrase Identification of Questions with Noisy Pretraining

bleu2vec: the Painfully Familiar Metric on Continuous Vector Space Steroids

Proceedings of the Second Workshop on Subword/Character LEvel Models

Contact Info

Product

Resources

About