Extrofitting: Enriching Word Representation and its Vector Space with Semantic Lexicons

Jo, Hwiyeol; Choi, Stanley Jungkyu

doi:10.48550/arxiv.1804.07946

Cited by 2 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Retrofitting could be specialized in semantic lexicons whereas extrofitting results in generalized word vectors (Jo and Choi, 2018). If then, we expect the results of retrofitting and extrofitting to complement each other.…”

Section: Extrofitting With Retrofittingmentioning

confidence: 93%

“…In this work, we first suggest unsupervised extrofitting, which enriches word vectors using semantically related words extracted from themselves instead of external semantic lexicons. The method uses Latent Semantic Analysis (LSA) (Landauer and Dumais, 1997) to extract semantically related words and then applies expansional retrofitting (extrofitting) (Jo and Choi, 2018) the word vectors with the information. Unsupervised extrofitting performs on par with extrofitting, which requires external semantic lexi-con.…”

Section: Introduction 1motivationmentioning

confidence: 99%

See 1 more Smart Citation

Expansional Retrofitting for Word Vector Enrichment

2018

Preprint

Self Cite

View full text Add to dashboard Cite

Retrofitting techniques, which inject external resources into word representations, have compensated the weakness of distributed representations in semantic and relational knowledge between words. Implicitly retrofitting word vectors by expansional technique outperforms retrofitting in word similarity tasks with word vector generalization. In this paper, we propose unsupervised extrofitting: expansional retrofitting (extrofitting) without external semantic lexicons. We also propose deep extrofitting: in-depth stacking of extrofitting and further combinations of extrofitting with retrofitting. When experimenting with GloVe, we show that our methods outperform the previous methods on most of word similarity tasks while requiring only synonyms as an external resource. Lastly, we show the effect of word vector enrichment on text classification task, as a downstream task.

show abstract

Section: Extrofitting With Retrofittingmentioning

confidence: 93%

Section: Introduction 1motivationmentioning

confidence: 99%

Expansional Retrofitting for Word Vector Enrichment

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Extrofitting Extrofitting (Jo and Choi, 2018) expands word embedding matrix W by concatenating W with r w :…”

Section: Preliminarymentioning

confidence: 99%

“…To sum up, we use the principle of vanilla extrofitting (Jo and Choi, 2018), which utilizes LDA algorithms to group in-domain instances (in this case, word vectors). Simultaneously, the algorithm pushes out-domain instances apart.…”

Section: Self-supervised Extrofittingmentioning

confidence: 99%

Self-supervised Post-processing Method to Enrich Pretrained Word Vectors

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Retrofitting techniques, which inject external resources into word representations, have compensated for the weakness of distributed representations in semantic and relational knowledge between words. However, the previous methods require additional external resources and strongly depend on the lexicon. To address the issues, we propose a simple extension of extrofitting, self-supervised extrofitting: extrofitting by its own word vector distribution. Our methods improve the vanilla embeddings on all of word similarity tasks without any external resources. Moreover, the method is also effective in various languages, which implies that our method will be useful in lexicon-scarce languages. As downstream tasks, we show its benefits in dialogue state tracking and text classification tasks, reporting better and generalized results compared to other word vector specialization methods. 1

show abstract

Extrofitting: Enriching Word Representation and its Vector Space with Semantic Lexicons

Cited by 2 publications

References 10 publications

Expansional Retrofitting for Word Vector Enrichment

Expansional Retrofitting for Word Vector Enrichment

Self-supervised Post-processing Method to Enrich Pretrained Word Vectors

Contact Info

Product

Resources

About