A Hybrid Learning Scheme for Chinese Word Embedding

Chen, Wenfan; Sheng, Weiguo

doi:10.18653/v1/w18-3011

Cited by 5 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For ideographic languages like ZH, word embeddings trained on stroke signals (which is analogous to subword information of alphabetic languages) achieve state-of-the-art performance (Cao et al, 2018), so we utilise them to obtain monolingual vectors. Compared with simplified characters (which dominate our training resources), traditional ones typically provide much richer stroke signals and thus benefit stroke-based embeddings (Chen and Sheng, 2018), e.g., traditional '葉' (leaf ) contains semantically related components of '艹' (plant) and '木' (wood), while its simplified version ('叶') does not.…”

Section: Methodsmentioning

confidence: 99%

Summarising Historical Text in Modern Languages

Peng,

Zheng,

Lin

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce the task of historical text summarisation, where documents in historical forms of a language are summarised in the corresponding modern language. This is a fundamentally important routine to historians and digital humanities researchers but has never been automated. We compile a high-quality gold-standard text summarisation dataset, which consists of historical German and Chinese news from hundreds of years ago summarised in modern German or Chinese. Based on cross-lingual transfer learning techniques, we propose a summarisation model that can be trained even with no cross-lingual (historical to modern) parallel data, and further benchmark it against state-of-the-art algorithms. We report automatic and human evaluations that distinguish the historic to modern language summarisation task from standard cross-lingual summarisation (i.e., modern to modern language), highlight the distinctness and value of our dataset, and demonstrate that our transfer learning approach outperforms standard cross-lingual benchmarks on this task.

show abstract

Section: Methodsmentioning

confidence: 99%

Summarising Historical Text in Modern Languages

Peng,

Zheng,

Lin

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The CBoW and SG models consider words as basic units ignoring rich subword information, thereby, significantly limiting the performance of the models [11]. This shortcoming was addressed in [1] by extending the SG model using subword information.…”

Section: Words Embeddingmentioning

confidence: 99%

hauWE: Hausa Words Embedding for Natural Language Processing

Abdulmumin

Galadanci

2019

2019 2nd International Conference of the IEEE Nigeria Computer Chapter (NigeriaComputConf)

View full text Add to dashboard Cite

Words embedding (distributed word vector representations) have become an essential component of many natural language processing (NLP) tasks such as machine translation, sentiment analysis, word analogy, named entity recognition and word similarity. Despite this, the only work that provides word vectors for Hausa language is that of [1] trained using fastText, consisting of only a few words vectors. This work presents words embedding models using Word2Vec's Continuous Bag of Words (CBoW) and Skip Gram (SG) models. The models, hauWE (Hausa Words Embedding), are bigger and better than the only previous model, making them more useful in NLP tasks. To compare the models, they were used to predict the 10 most similar words to 30 randomly selected Hausa words. hauWE CBoW's 88.7% and hauWE SG's 79.3% prediction accuracy greatly outperformed [1]'s 22.3%.

show abstract

“…Word representation is one of fundamental topics in Natural Language Processing (NLP). Many methods have been proposed to learn dense vector representations of words [1]- [4] and successfully applied in various tasks such as language modeling [5] and text classification [6].…”

Section: Introductionmentioning

confidence: 99%

Investigating Word Meta-Embeddings by Disentangling Common and Individual Information

et al. 2020

Self Cite

View full text Add to dashboard Cite

In the field of natural language processing, combining multiple pre-trained word embeddings has become a viable approach to improve word representations. However, there is still a lack of understanding of why such improvements can be achieved. In this paper, we investigate this issue by firstly proposing a novel word meta-embedding method. The proposed method tends to disentangle common and individual information from different word embeddings and learns representations for both. Based on the proposed method, we then carry out a systematic evaluation to provide a perspective on how common and individual information contributes to different tasks. Our intrinsic evaluation results suggest that common information is critical for word-level representations in terms of word similarity and relatedness. While, based on natural language inference, our extrinsic evaluation results show that common and individual information plays different roles and can complement each other. Further, both intrinsic and extrinsic evaluations reveal that explicitly separating common and individual information is beneficial for learning word meta-embeddings. INDEX TERMS Natural language processing, word meta-embedding, word representation.

show abstract

A Hybrid Learning Scheme for Chinese Word Embedding

Cited by 5 publications

References 13 publications

Summarising Historical Text in Modern Languages

Summarising Historical Text in Modern Languages

hauWE: Hausa Words Embedding for Natural Language Processing

Investigating Word Meta-Embeddings by Disentangling Common and Individual Information

Contact Info

Product

Resources

About