Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors

Jameel, Shoaib; Schockaert, Steven

doi:10.18653/v1/p19-1321

Cited by 5 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baselines. Our main baseline is the standard GloVe model, as in (1), which was found by Jameel and Schockaert (2019) to produce highly competitive entity embeddings, compared to a wide range of other methods. We also experimented with methods based on variational autoencoders, including the Neural Variational Document Model (Miao et al, 2016), but we were not able to obtain competitive results in this way.…”

Section: Methodsmentioning

confidence: 99%

A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

Alshaikh¹,

Bouraoui²,

Jeawak³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Various methods have already been proposed for learning entity embeddings from text descriptions. Such embeddings are commonly used for inferring properties of entities, for recommendation and entity-oriented search, and for injecting background knowledge into neural architectures, among others. Entity embeddings essentially serve as a compact encoding of a similarity relation, but similarity is an inherently multi-faceted notion. By representing entities as single vectors, existing methods leave it to downstream applications to identify these different facets, and to select the most relevant ones. In this paper, we propose a model that instead learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. We use a mixture-of-experts formulation to jointly learn these facet-specific embeddings. The individual entity embeddings are learned using a variant of the GloVe model, which has the advantage that we can easily identify which properties are modelled well in which of the learned embeddings. This is exploited by an associated gating network, which uses pre-trained word vectors to encourage the properties that are modelled by a given embedding to be semantically coherent, i.e. to encourage each of the individual embeddings to capture a meaningful facet.

show abstract

Section: Methodsmentioning

confidence: 99%

A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

Alshaikh¹,

Bouraoui²,

Jeawak³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another NMF-based method was proposed by Xu et al (2018), in which the Euclidean distance was substituted with Wasserstein distance. Jameel and Schockaert (2019) rewrote the NMF objective as a cumulative product of normal distributions, in which each factor is multiplied by a von Mises-Fisher (vMF) distribution of context word vectors, to hopefully cluster the context words since the vMF density retains the cosine similarity.…”

Section: Related Workmentioning

confidence: 99%

A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings

Zhu

Zhou

2020

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

We propose a novel generative model to explore both local and global context for joint learning topics and topic-specific word embeddings. In particular, we assume that global latent topics are shared across documents, a word is generated by a hidden semantic vector encoding its contextual semantic meaning, and its context words are generated conditional on both the hidden semantic vector and global latent topics. Topics are trained jointly with the word embeddings. The trained model maps words to topic-dependent embeddings, which naturally addresses the issue of word polysemy. Experimental results show that the proposed model outperforms the word-level embedding methods in both word similarity evaluation and word sense disambiguation. Furthermore, the model also extracts more coherent topics compared with existing neural topic models or other models for joint learning of topics and word embeddings. Finally, the model can be easily integrated with existing deep contextualized word embedding learning methods to further improve the performance of downstream tasks such as sentiment classification.

show abstract

Exploiting Explicit and Inferred Implicit Personas for Multi-turn Dialogue Generation

Wang

et al. 2021

Natural Language Processing and Chinese Computing

View full text Add to dashboard Cite

Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors

Cited by 5 publications

References 28 publications

A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings

Exploiting Explicit and Inferred Implicit Personas for Multi-turn Dialogue Generation

Contact Info

Product

Resources

About