Expansional Retrofitting for Word Vector Enrichment

Abstract

Retrofitting techniques, which inject external resources into word representations, have compensated the weakness of distributed representations in semantic and relational knowledge between words. Implicitly retrofitting word vectors by expansional technique outperforms retrofitting in word similarity tasks with word vector generalization. In this paper, we propose unsupervised extrofitting: expansional retrofitting (extrofitting) without external semantic lexicons. We also propose deep extrofitting: in-depth stacking of extrofitting and further combinations of extrofitting with retrofitting. When experimenting with GloVe, we show that our methods outperform the previous methods on most of word similarity tasks while requiring only synonyms as an external resource. Lastly, we show the effect of word vector enrichment on text classification task, as a downstream task.

Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings

“…We use GloVe (Pennington et al, 2014) glove.42B.300d as word embedding for M emb . We also perform word vector post-processing method, extrofitting (Jo, 2018), to improve the effect of initialization with pretrained word embeddings on text classification, as described in their paper. We use 4 text classification datasets; IMDB review (Maas et al, 2011), AGNews, Yelp review (Zhang et al, 2015), and Yahoo!Answers (Chang et al, 2008).…”