Continuous word embeddings for detecting local text reuses at the semantic level

Abstract

Text reuse is a common phenomenon in a variety of usergenerated content. Along with the quick expansion of social media, reuses of local text are occurring much more frequently than ever before. The task of detecting these local reuses serves as an essential step for many applications. It has attracted extensive attention in recent years. However, semantic level similarities have not received consideration in most previous works. In this paper, we introduce a novel method to efficiently detect local reuses at the semantic level for large scale problems. We propose to use continuous vector representations of words to capture the semantic level similarities between short text segments. In order to handle tens of billions of documents, methods based on information geometry and hashing methods are introduced to aggregate and map text segments presented by word embeddings to binary hash codes. Experimental results demonstrate that the proposed methods achieve significantly better performance than state-of-the-art approaches in all six document collections belonging to four different categories. At some recall levels, the precisions of the proposed method are even 10 times higher than previous methods. Moreover, the efficiency of the proposed method is comparable to or better than that of some other hashing methods.

Neural information retrieval: at the end of the early years

“…Gupta et al (2014) proposed an auto-encoder approach to mixed-script query expansion, considering transliterated search queries and learning a character-level ''topic'' joint distribution over features of both scripts. Zhang et al (2014) considered the task of local text reuse, with three annotators labeling whether or not a given passage represents reuse. New variants of DSSM were proposed.…”

“…Explicit The goal of Zhang et al (2014) is to efficiently retrieve passages that are semantically similar to a query, making use of hashing methods on word vectors that are learned in advance. Other than the given word vectors, no further deep learning is used.…”

Design and Develop Semantic Textual Document Clustering Model

“…Use PLSA to cipher word-topic distributions, fold in those distributions at the block level, and so choose segmentation points supported the similarity values of adjacent block pairs. (Sun, Li, Luo& Wu, 2008;Zhang, Kang, Qian& Huang, 2014;Rangel, Faria, Lima & Oliveira, 2016) use LDA on a corpus of segments, inter-segment cipher similarities via a Fisher kernel, and optimize segmentation via dynamic programming. (Misra, Yvon, Jose, & Cappe, 2009;Glavaš, Nanni & Ponzetto, 2016) use a document-level LDA model, treat sections as new documents and predict their LDA models, and so do segmentation via dynamic programming with probabilistic scores.…”

“…(Misra et al, 2009;Glavaš, Nanni & Ponzetto, 2016). The presence of logical structure clues within the document, scientific criteria and applied math similarity measures chiefly accustomed figure thematically coherent, contiguous text blocks in unstructured documents (Sun et al, 2008;Zhang et al, 2014;Rangel et al, 2016). Recent segmentation techniques have taken advantage of advances in generative topic modeling algorithms, which were specifically designed to spot issues at intervals text to cipher word-topic distributions (Lee, Han &Whang, 2007;Hung, Peng& Lee, 2015).…”

Learning Continuous Word Embedding with Metadata for Question Retrieval in Community Question Answering

“…Through looking up table (word embedding matrix) of M , the ith question q i can be represented by E w i = {e w ij , 1 ≤ j ≤ N i }, where e w ij is the word embedding of w ij . According to the framework of FK (Clinchant and Perronnin, 2013;Sanchez et al, 2013;Zhang et al, 2014b), questions are modeled by a probability density function. In this work, we use Gaussian mixture model (GMM) to do it.…”