An Empirical Evaluation of doc2vec with Practical Insights into
            Document Embedding Generation

Lau, Jey Han; Baldwin, Timothy

doi:10.18653/v1/w16-1609

Cited by 440 publications

(196 citation statements)

References 11 publications

Supporting

Mentioning

185

Contrasting

Unclassified

Order By: Relevance

“…However, no reports on the relevant literature describe an attempt to give meaning to each hidden node. One model of paragraph vectors (PV-DBOW) [5] uses pre-trained word embeddings that reportedly improve task performance [18]. Although this paper shows the possibility of learning proper document embeddings with good initialization of word embeddings, it does not demonstrate the possibility of interpretation of hidden nodes.…”

Section: Related Workmentioning

confidence: 94%

Semantically Readable Distributed Representation Learning and Its Expandability Using a Word Semantic Vector Dictionary

Keshi

Suzuki

Yoshino

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThe problem with distributed representations generated by neural networks is that the meaning of the features is difficult to understand. We propose a new method that gives a specific meaning to each node of a hidden layer by introducing a manually created word semantic vector dictionary into the initial weights and by using paragraph vector models. We conducted experiments to test the hypotheses using a single domain benchmark for Japanese Twitter sentiment analysis and then evaluated the expandability of the method using a diverse and large-scale benchmark. Moreover, we tested the domain-independence of the method using a Wikipedia corpus. Our experimental results demonstrated that the learned vector is better than the performance of the existing paragraph vector in the evaluation of the Twitter sentiment analysis task using the single domain benchmark. Also, we determined the readability of document embeddings, which means distributed representations of documents, in a user test. The definition of readability in this paper is that people can understand the meaning of large weighted features of distributed representations. A total of 52.4% of the top five weighted hidden nodes were related to tweets where one of the paragraph vector models learned the document embeddings. For the expandability evaluation of the method, we improved the dictionary based on the results of the hypothesis test and examined the relationship of the readability of learned word vectors and the task accuracy of Twitter sentiment analysis using the diverse and large-scale benchmark. We also conducted a word similarity task using the Wikipedia corpus to test the domain-independence of the method. We found the expandability results of the method are better than or comparable to the performance of the paragraph vector. Also, the objective and subjective evaluation support each hidden node maintaining a specific meaning. Thus, the proposed method succeeded in improving readability. key words: distributed representation, word semantic vector dictionary, paragraph vector, word2vec, Twitter, sentiment analysis

show abstract

Section: Related Workmentioning

confidence: 94%

Semantically Readable Distributed Representation Learning and Its Expandability Using a Word Semantic Vector Dictionary

Keshi

Suzuki

Yoshino

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…However, word embeddings came to the foreground by Mikolov, Chen, Corrado, and Dean (), who presented the popular Continuous Bag‐of‐Words model (CBOW) and the Continuous Skip‐gram model. Additionally, sentence embeddings (Doc2Vec (Lau & Baldwin, ) or Sent2vec (Pagliardini, Gupta, & Jaggi, )) as well as the popular GloVe (Global Vectors) (Pennington, Socher, & Manning, ) method are utilized by keyphrase extraction methods.…”

Section: Unsupervised Methodsmentioning

confidence: 99%

A review of keyphrase extraction

Papagiannopoulou

Tsoumakas

2019

WIREs Data Min & Knowl

133

View full text Add to dashboard Cite

Keyphrase extraction is a textual information processing task concerned with the automatic extraction of representative and characteristic phrases from a document that express all the key aspects of its content. Keyphrases constitute a succinct conceptual summary of a document, which is very useful in digital information management systems for semantic indexing, faceted search, document clustering and classification. This article introduces keyphrase extraction, provides a well‐structured review of the existing work, offers interesting insights on the different evaluation approaches, highlights open issues and presents a comparative experimental study of popular unsupervised techniques on five datasets. This article is categorized under: Ensemble Methods > Web Mining Ensemble Methods > Text Mining

show abstract

“…Word‐embedding approaches have been applied to various problems, such as text classification, text clustering, and textual similarity tasks. Text similarity can be measured through document or word embedding , . Recently, word‐embedding approaches have been mainly used for generating the features of the classifier, such as the support vector machine (SVM), the convolutional neural network (CNN), or the recurrent neural network (RNN) .…”

Section: Related Workmentioning

confidence: 99%

“…For a forum question duplication task and semantic textual similarity task, document‐embedding‐based approaches perform better than word‐embedding‐based approaches, while a PV‐DBOW‐based approach performs better than PV‐DM . Therefore, the proposed model utilizes the document‐embedding method PV‐DBOW to obtain a pre‐trained document vector.…”

Section: Learning Document Embedding With Training Setmentioning

confidence: 99%

“…Considering the characteristics of a multi-word text, such as a document, sentence, or paragraph, the authors of [24] proposed a document embedding method, called doc2vec, by modifying the word-embedding methods of Skipgram (PV-DBOW) and CBOW (PV-DM). For a forum question duplication task and semantic textual similarity task, document-embedding-based approaches perform better than word-embedding-based approaches, while a PV-DBOW-based approach performs better than PV-DM [18]. Therefore, the proposed model utilizes the document-embedding method PV-DBOW to obtain a pre-trained document vector.…”

Section: Skipgram and Doc2vecmentioning

confidence: 99%

See 1 more Smart Citation

Fine-Grained Mobile Application Clustering Model Using Retrofitted Document Embedding

Yoon¹,

Lee²,

Park³

et al. 2017

ETRI Journal

View full text Add to dashboard Cite

In this paper, we propose a fine-grained mobile application clustering model using retrofitted document embedding. To automatically determine the clusters and their numbers with no predefined categories, the proposed model initializes the clusters based on title keywords and then merges similar clusters. For improved clustering performance, the proposed model distinguishes between an accurate clustering step with titles and an expansive clustering step with descriptions. During the accurate clustering step, an automatically tagged set is constructed as a result. This set is utilized to learn a high-performance document vector. During the expansive clustering step, more applications are then classified using this document vector. Experimental results showed that the purity of the proposed model increased by 0.19, and the entropy decreased by 1.18, compared with the Kmeans algorithm. In addition, the mean average precision improved by more than 0.09 in a comparison with a support vector machine classifier.

show abstract

An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation

Cited by 440 publications

References 11 publications

Semantically Readable Distributed Representation Learning and Its Expandability Using a Word Semantic Vector Dictionary

Semantically Readable Distributed Representation Learning and Its Expandability Using a Word Semantic Vector Dictionary

A review of keyphrase extraction

Fine-Grained Mobile Application Clustering Model Using Retrofitted Document Embedding

Contact Info

Product

Resources

About