Learning Context-Specific Word/Character Embeddings

Abstract

Unsupervised word representations have demonstrated improvements in predictive generalization on various NLP tasks. Most of the existing models are in fact good at capturing the relatedness among words rather than their ''genuine'' similarity because the context representations are often represented by a sum (or an average) of the neighbor's embeddings, which simplifies the computation but ignores an important fact that the meaning of a word is determined by its context, reflecting not only the surrounding words but also the rules used to combine them (i.e. compositionality). On the other hand, much effort has been devoted to learning a single-prototype representation per word, which is problematic because many words are polysemous, and a single-prototype model is incapable of capturing phenomena of homonymy and polysemy. We present a neural network architecture to jointly learn word embeddings and context representations from large data sets. The explicitly produced context representations are further used to learn context-specific and multi-prototype word embeddings. Our embeddings were evaluated on several NLP tasks, and the experimental results demonstrated the proposed model outperformed other competitors and is applicable to intrinsically "character-based" languages.

RETRACTED: Multi-neural network-based sentiment analysis of food reviews based on character and word embeddings

“…The same word in Chinese may express different meanings in different words. 18 Because word embedding is out of the word, it cannot express the difference in different contexts, so it is not advisable to use the word embedding directly. The position of a single word in the entire word can be used as an index of the word in the entire word.…”

Chinese sentiment analysis model by integrating multi-granularity semantic features

“…Xu et al (2021) proposed a hybrid model by using BERT to capture the word vector with global sentence features, then using TextCNN to capture the local features. Chen et al (2015) decomposed Chinese words into characters and proposed a character-enhanced word embedding model.…”

„Mann“ is to “Donna” as「国王」is to « Reine » Adapting the Analogy Task for Multilingual and Contextual Embeddings

“…Datasets similar to BATS exist in Japanese and Icelandic (Karpinska et al, 2018;Friðriksdóttir et al, 2022), whereas GATS has been translated in Portuguese, Hindi, French, Polish, and Spanish (Hartmann et al, 2017;Grave et al, 2018;Cardellino, 2019). Other independently constructed datasets do exist (e.g., Venekoski and Vankka, 2017;Svoboda and Brychcín, 2018)crucially, covering all languages of interest to this study: in Chinese (Jin and Wu, 2012;Chen et al, 2015;, Dutch (Garneau et al, 2021), English (Turney 2008Mikolov et al 2013b, a.o. ), French (Grave et al, 2018), German (Köper et al, 2015), Italian (Berardi et al, 2015), and Spanish (Cardellino, 2019).…”