Retrieval of source documents in a text reuse system

Abstract

The architecture of the text-reuse detection system consists of three main modules, i.e., source retrieval, text analysis, and knowledge-based postprocessing. Each module plays an important role in the accuracy rate of the detection outputs. Therefore, this research focuses on developing the source retrieval system in cases where the source documents have been obfuscated in different levels. Two steps of term weighting were applied to get such documents. The first was the local-word weighting, which has been applied to the test or reused documents to select query per text segments. The tf-idf term weighting was applied for indexing all documents in the corpus and as the basis for computing cosine similarity between the queries per segment and the documents in the corpus. A two-step filtering technique was applied to get the source document candidates. Using artificial cases of text reuse testing, the system achieves the same rates of precision and recall that are 0.967, while the recall rate for the simulated cases of reused text is 0.66.

Pengaruh Synonym Recognition dalam Deteksi Kemiripan Teks Menggunakan Winnowing dan Cosine Similarity

“…The meaningless words are commonly referred to as stopwords. Some examples of stopwords are "juga," "dan," "untuk," and "adalah" [21]. It is necessary to delete these stopwords because if conjunctions frequently appear in a sentence, the text similarity percentage is very high, and it interferes with the accuracy of the text similarity method [22].…”

Comparation of Dice Similarity and Jaccard Coefficience Against Winnowing Algorithm For Similarity Detection of Indonesian Text Documents

“…The work process of IPD system is only based on the imitation of human expertise in recognizing parts of the text that experience a change in writing style as a sign of copy or paste text without comparing with other text[3]. EPD system process compares each document inputted with each document contained in the corpus to compare similarity [4]. Corpus must have several documents that have the same topic with the source of plagiarism to know the test of document similarity level.…”