Improving Word Representations with Document Labels

Yang, Liner; Chen, Xinxiong; Liu, Zhiyuan; Sun, Maosong

doi:10.1109/taslp.2017.2658019

Cited by 13 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By simply extending the Word2Vec structures, their objective function included the loss corresponding to global context. In addition, some fine-tuned models are proposed, they have integrated some other information, such as sentimental information [39], character information [22,39], document labels [40,41], syntactic information [42], on the basis of their original pre-trained vectors. In recent years, some new models have also been proposed, such as ELMo (Embeddings from Language Models) [43] and BERT (Bidirectional Encoder Representations from Transformer) [44].…”

Section: Word Embedding Modelsmentioning

confidence: 99%

Modeling multi-prototype Chinese word representation learning for word similarity

Yin

Wang

et al. 2021

Complex Intell. Syst.

View full text Add to dashboard Cite

The word similarity task is used to calculate the similarity of any pair of words, and is a basic technology of natural language processing (NLP). The existing method is based on word embedding, which fails to capture polysemy and is greatly influenced by the quality of the corpus. In this paper, we propose a multi-prototype Chinese word representation model (MP-CWR) for word similarity based on synonym knowledge base, including knowledge representation module and word similarity module. For the first module, we propose a dual attention to combine semantic information for jointly learning word knowledge representation. The MP-CWR model utilizes the synonyms as prior knowledge to supplement the relationship between words, which is helpful to solve the challenge of semantic expression due to insufficient data. As for the word similarity module, we propose a multi-prototype representation for each word. Then we calculate and fuse the conceptual similarity of two words to obtain the final result. Finally, we verify the effectiveness of our model on three public data sets with other baseline models. In addition, the experiments also prove the stability and scalability of our MP-CWR model under different corpora.

show abstract

Section: Word Embedding Modelsmentioning

confidence: 99%

Modeling multi-prototype Chinese word representation learning for word similarity

Yin

Wang

et al. 2021

Complex Intell. Syst.

View full text Add to dashboard Cite

show abstract

“…These models focus on Twitter sentiment classification and predict or rank sentiment polarity based on word embeddings in a fixed window of words across a sentence. Based on the Skip-Gram model, Zhang et al(2015) proposed a model for word-level and sentencelevel sentiment analysis, and Yang et al(2017) proposed a model that predicted the target word and its label simultaneously. Both of them took sentiment information as a part of the local context.…”

Section: Sentiment-specific Word Embeddingmentioning

confidence: 99%

“…These models predict or rank sentiment polarity based on word embeddings in a fixed window of words across a sentence. In addition, based on the Skip-Gram (Mikolov et al 2013), Zhang et al(2015) integrated the sentiment information by using the semantic word embeddings in the context to predict the sentiment polarity through a softmax layer, and Yang et al(2017) proposed a model that predicted the target word and its label simultaneously. Both of them took sentiment information as a part of the local context.…”

Section: Introductionmentioning

confidence: 99%

Learning Sentiment-Specific Word Embedding via Global Sentiment Representation

Lin

Yuan

et al. 2018

AAAI

View full text Add to dashboard Cite

Context-based word embedding learning approaches can model rich semantic and syntactic information. However, it is problematic for sentiment analysis because the words with similar contexts but opposite sentiment polarities, such as good and bad, are mapped into close word vectors in the embedding space. Recently, some sentiment embedding learning methods have been proposed, but most of them are designed to work well on sentence-level texts. Directly applying those models to document-level texts often leads to unsatisfied results. To address this issue, we present a sentiment-specific word embedding learning architecture that utilizes local context informationas well as global sentiment representation. The architecture is applicable for both sentence-level and document-level texts. We take global sentiment representation as a simple average of word embeddings in the text, and use a corruption strategy as a sentiment-dependent regularization. Extensive experiments conducted on several benchmark datasets demonstrate that the proposed architecture outperforms the state-of-the-art methods for sentiment classification.

show abstract

“…Recently, learning distributed representation of words, phrases, and sentences has gained a lot of a ention due to its applicability and superior performance over bag-of-words (BOW) features in a wide range of text processing tasks [6,14,25,[35][36][37]. ese models can be categorized into two groups: (i) task-agnostic or unsupervised models, and (ii) task-speci c or supervised models.…”

Section: Related Workmentioning

confidence: 99%

Regularized and Retrofitted models for Learning Sentence Representation with Context

Saha

Joty

Hassan

et al. 2017

Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representation has been used for a long time. In recent years, distributed representation of sentences learned by neural models from unlabeled data has been shown to outperform traditional bag-of-words representations. However, most existing methods belonging to the neural models consider only the content of a sentence, and disregard its relations with other sentences in the context. In this paper, we rst characterize two types of contexts depending on their scope and utility. We then propose two approaches to incorporate contextual information into content-based models. We evaluate our sentence representation models in a setup, where context is available to infer sentence vectors. Experimental results demonstrate that our proposed models outshine existing models on three fundamental tasks, such as, classifying, clustering, and ranking sentences.

show abstract

Improving Word Representations with Document Labels

Cited by 13 publications

References 19 publications

Modeling multi-prototype Chinese word representation learning for word similarity

Modeling multi-prototype Chinese word representation learning for word similarity

Learning Sentiment-Specific Word Embedding via Global Sentiment Representation

Regularized and Retrofitted models for Learning Sentence Representation with Context

Contact Info

Product

Resources

About