Enriching Word Vectors with Subword Information

Abstract

Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.

Assessing semantic similarity between concepts: A weighted‐feature‐based approach

“…The most widely used word embedding models are Word2Vec, 20 Global Vectors GloVe, 21 and FastText. 35 Word2Vec 20 perhaps is the most popular language model based on the neural network. Word2Vec has two architectures Skip-gram and Continuous Bag-of-Words (CBOW).…”

“…The main idea is to build word embedding of a word from its context words from a large corpus of text. The most widely used word embedding models are Word2Vec, Global Vectors GloVe, and FastText …”

Moliere

“…In order to do this, we turned to the UMLS SPECIALIST NLP toolset [1] as well as [16] and [9, 23]. Our process for constructing 𝒜 is summarized in Figure 2.…”

“…By using state-of-the-art tools, such as [16] and [9], we are able to find novel hypotheses without restricting the domain of our knowledge network or the resulting vocabulary when creating topics. As a result, is more generalized and yet still capable of identifying useful hypotheses.…”

“…is the most recent implementation of word2vec from Milkolov et al [30, 31, 23, 9]. Word2vec is a method which utilizes the skip-gram model to identify the relationships between words by analyzing word usage patterns.…”