SCDV : Sparse Composite Document Vectors using soft clustering over distributional representations

Abstract

We present a feature vector formation technique for documents -Sparse Composite Document Vector (SCDV) -which overcomes several shortcomings of the current distributional paragraph vector representations that are widely used for text representation. In SCDV, word embeddings are clustered to capture multiple semantic contexts in which words occur. They are then chained together to form document topic-vectors that can express complex, multi-topic documents. Through extensive experiments on multi-class and multi-label classification tasks, we outperform the previous state-of-the-art method, NTSG (Liu et al., 2015a). We also show that SCDV embeddings perform well on heterogeneous tasks like Topic Coherence, context-sensitive Learning and Information Retrieval. Moreover, we achieve significant reduction in training and prediction times compared to other representation methods. SCDV achieves best of both worlds -better performance with lower time and space complexity.

Comprehensive biological interpretation of gene signatures using semantic distributed representation

“…We defined proprietary functions using the packages published on the Comprehensive R Archive Network that can be used in the R language. In addition, we executed an original algorithm for creating a unique gene signature feature vector based on the sparse composite document vectors (SCDV) [9] method from NLP using only R language operations.…”

“…It should be noted that the original SCDV method of NLP, which is the basis of this method, can increase the speed and accuracy using the sparse method [9]. However, in gene signature analysis, the number of genes corresponding to the number of vocabularies is overwhelmingly small compared to natural language; thus, this step was excluded because the above procedure neither increased speed nor improved accuracy.…”

“…Beginning with Doc2Vec [8], which used a distributed representation of words, innovative techniques related to the distributed expression of a large number of sentences have been proposed in the past several years, and the accuracy of document interpretation has improved [9]. Typical methods of distributed representation of documents include statistical semantic extraction methods [10], methods that combine distributed representations of words [11] into document representations [12], methods that directly compress word and document IDs [8], methods of summing word vectors by multiplying the topics and specificities in the documents [9].…”

An AI-Based Methodology for the Automatic Classification of a Multiclass Ebook Collection Using Information From the Tables of Contents

“…Furthermore, a simple averaging of the TF-IDF weighting scheme of word vectors to produce document vectors has been investigated in [32]. The sparse composite document vector (SCDV), which was proposed in [33], extended the weighted averaging of word vectors from sentences to documents by using soft clustering over word vectors, while in [34], the approach was extended to capture also the multisense nature of words and to solve the problem of high dimensionality. This was realized by utilizing multisense word embeddings and by learning in a lowerdimensional manifold.…”

Picking Apart Story Salads

“…We choose k-medoids 7 (hereafter KM) as our clustering algorithm. For sentence embeddings, we experimented with (i) averaged 300D GloVe embeddings (Pennington et al, 2014), which have been shown to produce surprisingly strong performance in a variety of text classification tasks (Iyyer et al, 2015;Coates and Bollegala, 2018); (ii) skip-thought embeddings (Kiros et al, 2015); and (iii) SCDV (Mekala et al, 2017), a multisense-aware sentence embedding algorithm which builds upon pretrained GloVe embeddings using a Gaussian mixture model. Averaged GloVe embeddings gave the best performance in our experiments; to avoid clutter, we only report those results henceforth.…”