Radical-Enhanced Chinese Character Embedding

Sun, Yan; Lin, Lei; Yang, Nan; Ji, Zhenzhou; Wang, Xiaolong

doi:10.1007/978-3-319-12640-1_34

Cited by 103 publications

(68 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We learn an embedding for each character in the training corpus (Sun et al, 2014;.This removes the dependency on pre-processing the text, and better fits our intended use case: NER tagging over characters. Since there are many fewer characters than words, we learn many fewer embeddings.…”

Section: Character Embeddingsmentioning

confidence: 99%

Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings

Peng

Dredze

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

342

219

View full text Add to dashboard Cite

We consider the task of named entity recognition for Chinese social media. The long line of work in Chinese NER has focused on formal domains, and NER for social media has been largely restricted to English. We present a new corpus of Weibo messages annotated for both name and nominal mentions. Additionally, we evaluate three types of neural embeddings for representing Chinese text. Finally, we propose a joint training objective for the embeddings that makes use of both (NER) labeled and unlabeled raw text. Our methods yield a 9% improvement over a stateof-the-art baseline.

show abstract

Section: Character Embeddingsmentioning

confidence: 99%

Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings

Peng

Dredze

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

342

219

View full text Add to dashboard Cite

show abstract

“…Incorporating subword information for word embeddings (Bojanowski et al, 2017;Cotterell et al, 2016b;Wieting et al, 2016;Yin et al, 2016) facilitates modeling rare words and can improve the performance of several NLP tasks to which the embeddings are applied. Besides, people also consider character embeddings which have been utilized in Chinese word segmentation (Sun et al, 2014).…”

Section: Related Workmentioning

confidence: 99%

Incorporating Chinese Characters of Words for Lexical Sememe Prediction

Jin¹,

Zhu

Liu

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Sememes are minimum semantic units of concepts in human languages, such that each word sense is composed of one or multiple sememes. Words are usually manually annotated with their sememes by linguists, and form linguistic commonsense knowledge bases widely used in various NLP tasks. Recently, the lexical sememe prediction task has been introduced. It consists of automatically recommending sememes for words, which is expected to improve annotation efficiency and consistency. However, existing methods of lexical sememe prediction typically rely on the external context of words to represent the meaning, which usually fails to deal with low-frequency and out-ofvocabulary words. To address this issue for Chinese, we propose a novel framework to take advantage of both internal character information and external context information of words. We experiment on HowNet, a Chinese sememe knowledge base, and demonstrate that our framework outperforms state-of-the-art baselines by a large margin, and maintains a robust performance even for low-frequency words. i

show abstract

“…With regards to ideographic languages, there is work in information retrieval that has considered the appropriate representation for indexing; the focus has typically been word versus character (Kwok, 1997;Baldwin, 2009), but Fujii and Croft (1993) considered (though ultimately rejected) subcharacter based indexing. In terms of investigations of the usefulness of sub-character representations for neural network models in ideographic languages, relevant work includes recent papers that use sub-character information to assist in the training of character embeddings for Chinese (Sun et al, 2014;Li et al, 2015;Yin et al, 2016) or build sub-character embeddings directly (Shi et al, 2015), demonstrating that sub-character information is useful for representing semantics in Chinese. However, our work differs not only in language and task, but also in our use of decomposition, since the work done in Chinese has primarily focused on a single semantically relevant sub-character (known as the radical), despite the fact that other sub-characters do provide additional semantic information in some characters.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the First Workshop on Subword and Character Level Models in NLP

2017

View full text Add to dashboard Cite

IntroductionTraditional NLP starts with a hand-engineered layer of representation, the level of tokens or words. A tokenization component first breaks up the text into units using manually designed rules. Tokens are then processed by components such as word segmentation, morphological analysis and multiword recognition. The heterogeneity of these components makes it hard to create integrated models of both structure within tokens (e.g., morphology) and structure across multiple tokens (e.g., multi-word expressions). This approach can perform poorly (i) for morphologically rich languages, (ii) for noisy text, (iii) for languages in which the recognition of words is difficult and (iv) for adaptation to new domains; and (v) it can impede the optimization of preprocessing in end-to-end learning.The workshop provides a forum for discussing recent advances as well as future directions on sub-word and character-level natural language processing and representation learning that address these problems.We received 37 submissions, out of which we accepted 24 as papers and 4 as extended abstracts. AbstractMost of neural language models use different kinds of embeddings for word prediction. While word embeddings can be associated to each word in the vocabulary or derived from characters as well as factored morphological decomposition, these word representations are mainly used to parametrize the input, i.e. the context of prediction. This work investigates the effect of using subword units (character and factored morphological decomposition) to build output representations for neural language modeling. We present a case study on Czech, a morphologically-rich language, experimenting with different input and output representations. When working with the full training vocabulary, despite unstable training, our experiments show that augmenting the output word representations with character-based embeddings can significantly improve the performance of the model. Moreover, reducing the size of the output look-up table, to let the character-based embeddings represent rare words, brings further improvement. IntroductionMost of neural language models, such as n-gram models (Bengio et al., 2003) are word based and rely on the definition of a finite vocabulary V. Therefore, a look-up table maps each wordw ∈ V to a vector of real features, and is stored in a matrix. While this approach yields significant improvement for a variety of tasks and languages, see for instance (Schwenk, 2007) in speech recognition and (Le et al., 2012; Devlin et al., 2014; in machine translation, it induces several limitations.For morphologically-rich languages, like Czech or German, the lexical coverage is still an important issue, since there is a combinatorial explosion of word forms, most of which are hardly observed on training data. On the one hand, growing the look-up table is not a solution, since it would increase the number of parameters without having enough training examples for a proper estimation. On the other hand, rare words can be replaced...

show abstract

Radical-Enhanced Chinese Character Embedding

Cited by 103 publications

References 13 publications

Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings

Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings

Incorporating Chinese Characters of Words for Lexical Sememe Prediction

Proceedings of the First Workshop on Subword and Character Level Models in NLP

Contact Info

Product

Resources

About